[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 15:08:14.332661 12707 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.254s user 0.454s sys 0.798s
W20260924 15:08:14.333468 12707 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.255s user 0.454s sys 0.798s
I20260924 15:08:14.337118 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:43037
I20260924 15:08:14.340133 12707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 15:08:14.342221 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:14.363001 12718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:14.364363 12719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:14.476147 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:14.476058 12721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:14.477478 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:14.477736 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:14.477983 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262494477964 us; error 0 us; skew 500 ppm
I20260924 15:08:14.478930 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:14.485523 12707 webserver.cc:533] Webserver started at http://127.12.104.254:41305/ using document root <none> and password file <none>
I20260924 15:08:14.486644 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:14.486929 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:14.487502 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:14.496183 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "e494456e7f244df189a6e26abc400244"
format_stamp: "Formatted at 2026-09-24 15:08:14 on dist-test-slave-9164"
I20260924 15:08:14.505972 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.005s sys 0.005s
I20260924 15:08:14.512642 12726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:14.514698 12707 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260924 15:08:14.515472 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "e494456e7f244df189a6e26abc400244"
format_stamp: "Formatted at 2026-09-24 15:08:14 on dist-test-slave-9164"
I20260924 15:08:14.515888 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:14.545289 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:14.547165 12707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 15:08:14.547765 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:14.635250 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:43037
I20260924 15:08:14.635308 12777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:43037 every 8 connection(s)
I20260924 15:08:14.642100 12778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:14.666833 12778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: Bootstrap starting.
I20260924 15:08:14.674089 12778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:14.676442 12778 log.cc:826] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: Log is configured to *not* fsync() on all Append() calls
I20260924 15:08:14.681929 12778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: No bootstrap required, opened a new log
I20260924 15:08:14.706986 12778 raft_consensus.cc:359] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER }
I20260924 15:08:14.707721 12778 raft_consensus.cc:385] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:14.708124 12778 raft_consensus.cc:740] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494456e7f244df189a6e26abc400244, State: Initialized, Role: FOLLOWER
I20260924 15:08:14.709668 12778 consensus_queue.cc:260] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER }
I20260924 15:08:14.710386 12778 raft_consensus.cc:399] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:08:14.710850 12778 raft_consensus.cc:493] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:08:14.711373 12778 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:14.717360 12778 raft_consensus.cc:515] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER }
I20260924 15:08:14.718515 12778 leader_election.cc:304] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e494456e7f244df189a6e26abc400244; no voters:
I20260924 15:08:14.718669 12781 raft_consensus.cc:493] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:14.719255 12781 raft_consensus.cc:515] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER }
I20260924 15:08:14.721072 12778 leader_election.cc:290] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:08:14.721697 12782 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:14.724918 12781 leader_election.cc:304] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e494456e7f244df189a6e26abc400244; no voters:
I20260924 15:08:14.725404 12782 raft_consensus.cc:697] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 LEADER]: Becoming Leader. State: Replica: e494456e7f244df189a6e26abc400244, State: Running, Role: LEADER
I20260924 15:08:14.726496 12782 consensus_queue.cc:237] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER }
I20260924 15:08:14.727481 12778 sys_catalog.cc:565] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:14.727986 12781 leader_election.cc:290] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260924 15:08:14.735328 12783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e494456e7f244df189a6e26abc400244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER } }
I20260924 15:08:14.735332 12784 sys_catalog.cc:455] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e494456e7f244df189a6e26abc400244. Latest consensus state: current_term: 1 leader_uuid: "e494456e7f244df189a6e26abc400244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e494456e7f244df189a6e26abc400244" member_type: VOTER } }
I20260924 15:08:14.736174 12783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:14.736366 12784 sys_catalog.cc:458] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:14.742831 12791 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:14.758862 12791 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:14.760957 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:14.780434 12791 catalog_manager.cc:1396] Generated new cluster ID: 6e74a59deaeb4ca0b8899d9c683f5923
I20260924 15:08:14.780787 12791 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:14.801904 12791 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:14.804543 12791 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:14.825402 12791 catalog_manager.cc:6219] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: Generated new TSK 0
I20260924 15:08:14.826815 12791 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:14.893208 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:14.904336 12803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:14.906044 12804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:14.910080 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:14.910050 12806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:14.911311 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:14.911594 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:14.911845 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262494911804 us; error 0 us; skew 500 ppm
I20260924 15:08:14.912537 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:14.917851 12707 webserver.cc:533] Webserver started at http://127.12.104.193:45383/ using document root <none> and password file <none>
I20260924 15:08:14.918576 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:14.919219 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:14.919811 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:14.921483 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "9039862d1386461a8f08065d9522c93c"
format_stamp: "Formatted at 2026-09-24 15:08:14 on dist-test-slave-9164"
I20260924 15:08:14.929020 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.005s
I20260924 15:08:14.935110 12811 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:14.936350 12707 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260924 15:08:14.936743 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "9039862d1386461a8f08065d9522c93c"
format_stamp: "Formatted at 2026-09-24 15:08:14 on dist-test-slave-9164"
I20260924 15:08:14.937134 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:14.959671 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:14.961215 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:14.964646 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:14.971073 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:14.971490 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:14.971846 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:14.972019 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:15.096128 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:42697
I20260924 15:08:15.096218 12874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:42697 every 8 connection(s)
I20260924 15:08:15.105235 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:15.116922 12879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:15.123530 12880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:15.132688 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:15.132591 12882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:15.134137 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:15.134498 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:15.134783 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262495134758 us; error 0 us; skew 500 ppm
I20260924 15:08:15.135582 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:15.137717 12875 heartbeater.cc:344] Connected to a master server at 127.12.104.254:43037
I20260924 15:08:15.138269 12875 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:15.139663 12707 webserver.cc:533] Webserver started at http://127.12.104.194:33493/ using document root <none> and password file <none>
I20260924 15:08:15.139623 12875 heartbeater.cc:507] Master 127.12.104.254:43037 requested a full tablet report, sending...
I20260924 15:08:15.140391 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:15.140669 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:15.140998 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:15.142750 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "10b3b358106b42398a5d2a3f0cb51436"
format_stamp: "Formatted at 2026-09-24 15:08:15 on dist-test-slave-9164"
I20260924 15:08:15.144356 12743 ts_manager.cc:194] Registered new tserver with Master: 9039862d1386461a8f08065d9522c93c (127.12.104.193:42697)
I20260924 15:08:15.147902 12743 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47968
I20260924 15:08:15.150558 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260924 15:08:15.157490 12887 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:15.158951 12707 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260924 15:08:15.159550 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "10b3b358106b42398a5d2a3f0cb51436"
format_stamp: "Formatted at 2026-09-24 15:08:15 on dist-test-slave-9164"
I20260924 15:08:15.160182 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:15.174016 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:15.175537 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:15.178882 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:15.181620 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:15.181843 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:15.182183 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:15.182401 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260924 15:08:15.253157 12875 heartbeater.cc:499] Master 127.12.104.254:43037 was elected leader, sending a full tablet report...
I20260924 15:08:15.293987 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:43219
I20260924 15:08:15.294103 12950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:43219 every 8 connection(s)
I20260924 15:08:15.313218 12951 heartbeater.cc:344] Connected to a master server at 127.12.104.254:43037
I20260924 15:08:15.313874 12951 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:15.315057 12951 heartbeater.cc:507] Master 127.12.104.254:43037 requested a full tablet report, sending...
I20260924 15:08:15.318183 12743 ts_manager.cc:194] Registered new tserver with Master: 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194:43219)
I20260924 15:08:15.318601 12707 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.01907203s
I20260924 15:08:15.320250 12743 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47978
I20260924 15:08:15.355165 12743 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260924 15:08:15.378086 12743 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260924 15:08:15.424436 12951 heartbeater.cc:499] Master 127.12.104.254:43037 was elected leader, sending a full tablet report...
I20260924 15:08:15.453610 12915 tablet_service.cc:1511] Processing CreateTablet for tablet 279290e600514cc8b8442f8ca0c6db71 (DEFAULT_TABLE table=test-table [id=db3dae6f1c154322b1c322604d80b6ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:15.454665 12914 tablet_service.cc:1511] Processing CreateTablet for tablet 2d57b829404145b6abcc13c16ef091ac (DEFAULT_TABLE table=test-table [id=db3dae6f1c154322b1c322604d80b6ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:15.455519 12915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 279290e600514cc8b8442f8ca0c6db71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:15.456205 12914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d57b829404145b6abcc13c16ef091ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:15.459553 12838 tablet_service.cc:1511] Processing CreateTablet for tablet 2d57b829404145b6abcc13c16ef091ac (DEFAULT_TABLE table=test-table [id=db3dae6f1c154322b1c322604d80b6ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:15.461180 12838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d57b829404145b6abcc13c16ef091ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:15.462772 12839 tablet_service.cc:1511] Processing CreateTablet for tablet 279290e600514cc8b8442f8ca0c6db71 (DEFAULT_TABLE table=test-table [id=db3dae6f1c154322b1c322604d80b6ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:15.464601 12839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 279290e600514cc8b8442f8ca0c6db71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:15.498515 12967 tablet_bootstrap.cc:492] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: Bootstrap starting.
I20260924 15:08:15.503767 12968 tablet_bootstrap.cc:492] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: Bootstrap starting.
I20260924 15:08:15.508716 12967 tablet_bootstrap.cc:654] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:15.510358 12968 tablet_bootstrap.cc:654] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:15.515895 12967 tablet_bootstrap.cc:492] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: No bootstrap required, opened a new log
I20260924 15:08:15.516455 12967 ts_tablet_manager.cc:1403] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260924 15:08:15.516969 12968 tablet_bootstrap.cc:492] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: No bootstrap required, opened a new log
I20260924 15:08:15.517424 12968 ts_tablet_manager.cc:1403] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260924 15:08:15.520390 12968 raft_consensus.cc:359] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.521008 12968 raft_consensus.cc:740] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9039862d1386461a8f08065d9522c93c, State: Initialized, Role: FOLLOWER
I20260924 15:08:15.520805 12967 raft_consensus.cc:359] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.521608 12967 raft_consensus.cc:740] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Initialized, Role: FOLLOWER
I20260924 15:08:15.521684 12968 consensus_queue.cc:260] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.522368 12967 consensus_queue.cc:260] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.526525 12967 ts_tablet_manager.cc:1434] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260924 15:08:15.527865 12967 tablet_bootstrap.cc:492] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: Bootstrap starting.
I20260924 15:08:15.534890 12967 tablet_bootstrap.cc:654] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:15.544443 12968 ts_tablet_manager.cc:1434] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: Time spent starting tablet: real 0.027s user 0.010s sys 0.016s
I20260924 15:08:15.545655 12968 tablet_bootstrap.cc:492] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: Bootstrap starting.
W20260924 15:08:15.550503 12952 tablet.cc:2433] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 15:08:15.553145 12968 tablet_bootstrap.cc:654] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:15.570158 12968 tablet_bootstrap.cc:492] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: No bootstrap required, opened a new log
I20260924 15:08:15.570772 12968 ts_tablet_manager.cc:1403] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.006s
I20260924 15:08:15.580416 12967 tablet_bootstrap.cc:492] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: No bootstrap required, opened a new log
I20260924 15:08:15.581043 12967 ts_tablet_manager.cc:1403] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: Time spent bootstrapping tablet: real 0.054s user 0.026s sys 0.020s
I20260924 15:08:15.581055 12968 raft_consensus.cc:359] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.582039 12968 raft_consensus.cc:740] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9039862d1386461a8f08065d9522c93c, State: Initialized, Role: FOLLOWER
I20260924 15:08:15.582785 12968 consensus_queue.cc:260] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.585386 12968 ts_tablet_manager.cc:1434] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260924 15:08:15.585310 12967 raft_consensus.cc:359] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.586266 12967 raft_consensus.cc:740] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Initialized, Role: FOLLOWER
I20260924 15:08:15.586969 12967 consensus_queue.cc:260] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.590848 12967 ts_tablet_manager.cc:1434] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: Time spent starting tablet: real 0.009s user 0.001s sys 0.007s
I20260924 15:08:15.646085 12707 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260924 15:08:15.646520 12707 raft_consensus.cc:423] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Emulating election...
I20260924 15:08:15.646863 12707 raft_consensus.cc:3060] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:15.653469 12707 raft_consensus.cc:697] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Becoming Leader. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:15.654274 12707 consensus_queue.cc:237] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:15.657696 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:15.667506 12743 catalog_manager.cc:5838] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194). New cstate: current_term: 1 leader_uuid: "10b3b358106b42398a5d2a3f0cb51436" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:15.687572 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:15.793200 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:15.997835 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:16.175812 12849 raft_consensus.cc:3060] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:16.184839 12978 consensus_queue.cc:1048] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 15:08:16.303740 12707 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260924 15:08:16.304103 12707 raft_consensus.cc:423] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Emulating election...
I20260924 15:08:16.304405 12707 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:16.311333 12707 raft_consensus.cc:697] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Becoming Leader. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:16.312156 12707 consensus_queue.cc:237] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:16.317543 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:16.320063 12741 catalog_manager.cc:5838] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194). New cstate: current_term: 1 leader_uuid: "10b3b358106b42398a5d2a3f0cb51436" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:16.323071 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:16.428248 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:16.634589 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260924 15:08:16.714874 12849 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:16.721315 12982 consensus_queue.cc:1048] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:17.141582 12707 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260924 15:08:17.142045 12707 raft_consensus.cc:423] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Emulating election...
I20260924 15:08:17.142385 12707 raft_consensus.cc:3055] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 15:08:17.142620 12707 raft_consensus.cc:740] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:17.143317 12707 consensus_queue.cc:260] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:17.144452 12707 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:17.150023 12707 raft_consensus.cc:697] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 2 LEADER]: Becoming Leader. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:17.150820 12707 consensus_queue.cc:237] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:17.152281 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:17.156381 12741 catalog_manager.cc:5838] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "10b3b358106b42398a5d2a3f0cb51436" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:17.157246 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260924 15:08:17.264066 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260924 15:08:17.469905 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260924 15:08:17.586091 12849 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:17.592208 12849 raft_consensus.cc:1275] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 2 FOLLOWER]: Refusing update from remote peer 10b3b358106b42398a5d2a3f0cb51436: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 15:08:17.593621 12991 consensus_queue.cc:1048] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 15:08:17.774920 12707 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260924 15:08:17.775382 12707 raft_consensus.cc:423] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 1 FOLLOWER]: Emulating election...
I20260924 15:08:17.775688 12707 raft_consensus.cc:3060] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:17.783098 12707 raft_consensus.cc:697] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 2 LEADER]: Becoming Leader. State: Replica: 9039862d1386461a8f08065d9522c93c, State: Running, Role: LEADER
I20260924 15:08:17.784287 12707 consensus_queue.cc:237] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:17.787051 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:17.791076 12742 catalog_manager.cc:5838] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c reported cstate change: term changed from 1 to 2, leader changed from 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194) to 9039862d1386461a8f08065d9522c93c (127.12.104.193). New cstate: current_term: 2 leader_uuid: "9039862d1386461a8f08065d9522c93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: UNKNOWN } } }
I20260924 15:08:17.792009 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260924 15:08:17.897085 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260924 15:08:18.082078 12849 raft_consensus.cc:1240] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 2 LEADER]: Rejecting Update request from peer 10b3b358106b42398a5d2a3f0cb51436 for earlier term 1. Current term is 2. Ops: []
I20260924 15:08:18.084165 12991 consensus_queue.cc:1059] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 15:08:18.085021 12991 consensus_queue.cc:1243] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Peer 9039862d1386461a8f08065d9522c93c log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260924 15:08:18.085578 12998 raft_consensus.cc:3055] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 15:08:18.086090 12998 raft_consensus.cc:740] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:18.086808 12998 consensus_queue.cc:260] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:18.087687 12998 raft_consensus.cc:3060] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:18.103438 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260924 15:08:18.184890 12925 raft_consensus.cc:1275] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 2 FOLLOWER]: Refusing update from remote peer 9039862d1386461a8f08065d9522c93c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 15:08:18.186474 12997 consensus_queue.cc:1048] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 15:08:18.609035 12707 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260924 15:08:18.609350 12707 raft_consensus.cc:423] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 2 FOLLOWER]: Emulating election...
I20260924 15:08:18.609761 12707 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:18.614711 12707 raft_consensus.cc:697] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 3 LEADER]: Becoming Leader. State: Replica: 9039862d1386461a8f08065d9522c93c, State: Running, Role: LEADER
I20260924 15:08:18.615594 12707 consensus_queue.cc:237] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:18.617920 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:18.622805 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260924 15:08:18.622370 12741 catalog_manager.cc:5838] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c reported cstate change: term changed from 2 to 3, leader changed from 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194) to 9039862d1386461a8f08065d9522c93c (127.12.104.193). New cstate: current_term: 3 leader_uuid: "9039862d1386461a8f08065d9522c93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: UNKNOWN } } }
I20260924 15:08:18.727488 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260924 15:08:18.933585 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260924 15:08:19.064169 12925 raft_consensus.cc:3055] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 15:08:19.064538 12925 raft_consensus.cc:740] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:19.065037 12925 consensus_queue.cc:260] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:19.066421 12925 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:19.072412 12925 raft_consensus.cc:1275] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 3 FOLLOWER]: Refusing update from remote peer 9039862d1386461a8f08065d9522c93c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 15:08:19.073923 13008 consensus_queue.cc:1048] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 15:08:19.238695 12707 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260924 15:08:19.238994 12707 raft_consensus.cc:423] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 3 FOLLOWER]: Emulating election...
I20260924 15:08:19.239252 12707 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 3 FOLLOWER]: Advancing to term 4
I20260924 15:08:19.243533 12707 raft_consensus.cc:697] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 4 LEADER]: Becoming Leader. State: Replica: 10b3b358106b42398a5d2a3f0cb51436, State: Running, Role: LEADER
I20260924 15:08:19.244220 12707 consensus_queue.cc:237] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:19.246073 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:19.250267 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260924 15:08:19.250432 12743 catalog_manager.cc:5838] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 reported cstate change: term changed from 3 to 4, leader changed from 9039862d1386461a8f08065d9522c93c (127.12.104.193) to 10b3b358106b42398a5d2a3f0cb51436 (127.12.104.194). New cstate: current_term: 4 leader_uuid: "10b3b358106b42398a5d2a3f0cb51436" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:19.355878 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260924 15:08:19.535772 12925 raft_consensus.cc:1240] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 4 LEADER]: Rejecting Update request from peer 9039862d1386461a8f08065d9522c93c for earlier term 3. Current term is 4. Ops: []
I20260924 15:08:19.537497 13017 consensus_queue.cc:1059] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 15:08:19.538275 13017 consensus_queue.cc:1243] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [LEADER]: Peer 10b3b358106b42398a5d2a3f0cb51436 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260924 15:08:19.538722 13008 raft_consensus.cc:3055] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 3 LEADER]: Stepping down as leader of term 3
I20260924 15:08:19.538981 13008 raft_consensus.cc:740] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9039862d1386461a8f08065d9522c93c, State: Running, Role: LEADER
I20260924 15:08:19.539589 13008 consensus_queue.cc:260] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 } } peers { permanent_uuid: "10b3b358106b42398a5d2a3f0cb51436" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43219 } }
I20260924 15:08:19.540393 13008 raft_consensus.cc:3060] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 3 FOLLOWER]: Advancing to term 4
I20260924 15:08:19.562427 12707 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260924 15:08:19.663522 12849 raft_consensus.cc:1275] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 4 FOLLOWER]: Refusing update from remote peer 10b3b358106b42398a5d2a3f0cb51436: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 15:08:19.665159 13015 consensus_queue.cc:1048] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9039862d1386461a8f08065d9522c93c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 15:08:20.074249 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:08:20.104799 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:20.105580 12707 tablet_replica.cc:333] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c: stopping tablet replica
I20260924 15:08:20.106191 12707 raft_consensus.cc:2243] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 2 LEADER]: Raft consensus shutting down.
I20260924 15:08:20.106917 12707 raft_consensus.cc:2272] T 2d57b829404145b6abcc13c16ef091ac P 9039862d1386461a8f08065d9522c93c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:20.108944 12707 tablet_replica.cc:333] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c: stopping tablet replica
I20260924 15:08:20.109366 12707 raft_consensus.cc:2243] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 15:08:20.109725 12707 raft_consensus.cc:2272] T 279290e600514cc8b8442f8ca0c6db71 P 9039862d1386461a8f08065d9522c93c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:20.211885 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:08:20.223624 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:08:20.244946 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:20.245610 12707 tablet_replica.cc:333] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436: stopping tablet replica
I20260924 15:08:20.246073 12707 raft_consensus.cc:2243] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 15:08:20.246555 12707 raft_consensus.cc:2272] T 2d57b829404145b6abcc13c16ef091ac P 10b3b358106b42398a5d2a3f0cb51436 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:20.248101 12707 tablet_replica.cc:333] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436: stopping tablet replica
I20260924 15:08:20.248520 12707 raft_consensus.cc:2243] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 4 LEADER]: Raft consensus shutting down.
I20260924 15:08:20.249049 12707 raft_consensus.cc:2272] T 279290e600514cc8b8442f8ca0c6db71 P 10b3b358106b42398a5d2a3f0cb51436 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:20.269833 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:08:20.278872 12707 master.cc:562] Master@127.12.104.254:43037 shutting down...
I20260924 15:08:20.429550 12801 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260924 15:08:20.474920 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:20.475570 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:20.475930 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P e494456e7f244df189a6e26abc400244: stopping tablet replica
I20260924 15:08:20.498131 12707 master.cc:584] Master@127.12.104.254:43037 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7497 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260924 15:08:20.529772 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:33723
I20260924 15:08:20.531066 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:20.537503 13030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.538758 13031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.540602 13033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:20.542683 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:20.543943 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:20.544217 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:20.544446 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262500544424 us; error 0 us; skew 500 ppm
I20260924 15:08:20.545184 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:20.548916 12707 webserver.cc:533] Webserver started at http://127.12.104.254:36315/ using document root <none> and password file <none>
I20260924 15:08:20.549741 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:20.550037 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:20.550436 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:20.551716 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "88f4142c5cac44589515bfb3afce03e1"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.558700 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260924 15:08:20.563589 13038 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.564889 12707 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260924 15:08:20.565212 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "88f4142c5cac44589515bfb3afce03e1"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.565574 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:20.588869 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:20.589948 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:20.634214 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:33723
I20260924 15:08:20.634377 13089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:33723 every 8 connection(s)
I20260924 15:08:20.638810 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:20.649127 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1: Bootstrap starting.
I20260924 15:08:20.653425 13090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:20.657721 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1: No bootstrap required, opened a new log
I20260924 15:08:20.659989 13090 raft_consensus.cc:359] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER }
I20260924 15:08:20.660388 13090 raft_consensus.cc:385] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:20.660681 13090 raft_consensus.cc:740] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f4142c5cac44589515bfb3afce03e1, State: Initialized, Role: FOLLOWER
I20260924 15:08:20.661304 13090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER }
I20260924 15:08:20.661798 13090 raft_consensus.cc:399] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:08:20.662058 13090 raft_consensus.cc:493] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:08:20.662408 13090 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:20.667601 13090 raft_consensus.cc:515] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER }
I20260924 15:08:20.668236 13090 leader_election.cc:304] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88f4142c5cac44589515bfb3afce03e1; no voters:
I20260924 15:08:20.669682 13090 leader_election.cc:290] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:08:20.670030 13094 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:20.671819 13094 raft_consensus.cc:697] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 1 LEADER]: Becoming Leader. State: Replica: 88f4142c5cac44589515bfb3afce03e1, State: Running, Role: LEADER
I20260924 15:08:20.672564 13094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER }
I20260924 15:08:20.672956 13090 sys_catalog.cc:565] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:20.675688 13095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88f4142c5cac44589515bfb3afce03e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER } }
I20260924 15:08:20.675796 13096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88f4142c5cac44589515bfb3afce03e1. Latest consensus state: current_term: 1 leader_uuid: "88f4142c5cac44589515bfb3afce03e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f4142c5cac44589515bfb3afce03e1" member_type: VOTER } }
I20260924 15:08:20.676451 13095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:20.676695 13096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:20.678675 13099 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:20.684605 13099 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:20.692991 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:20.696445 13099 catalog_manager.cc:1396] Generated new cluster ID: 0d6fb4ecfa834903bdec6bbd056a35f0
I20260924 15:08:20.696782 13099 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:20.720657 13099 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:20.722208 13099 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:20.734802 13099 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1: Generated new TSK 0
I20260924 15:08:20.735448 13099 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:20.762657 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:20.769336 13112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.770380 13113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.772181 13115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:20.773166 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:20.773980 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:20.774189 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:20.774462 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262500774443 us; error 0 us; skew 500 ppm
I20260924 15:08:20.774943 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:20.777280 12707 webserver.cc:533] Webserver started at http://127.12.104.193:45505/ using document root <none> and password file <none>
I20260924 15:08:20.777722 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:20.777878 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:20.778113 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:20.779194 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "592c9b1c25db4f19aabd16f5640a961b"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.784052 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 15:08:20.787604 13120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.788532 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260924 15:08:20.788930 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "592c9b1c25db4f19aabd16f5640a961b"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.789232 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:20.809975 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:20.811213 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:20.812827 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:20.815291 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:20.815546 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.815820 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:20.816002 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.856745 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:41297
I20260924 15:08:20.856868 13182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:41297 every 8 connection(s)
I20260924 15:08:20.861819 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:20.869689 13187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.872083 13188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:20.875576 13190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:20.875720 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:20.876722 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:20.876919 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:20.877048 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262500877036 us; error 0 us; skew 500 ppm
I20260924 15:08:20.877521 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:20.879976 12707 webserver.cc:533] Webserver started at http://127.12.104.194:40081/ using document root <none> and password file <none>
I20260924 15:08:20.880419 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:20.880574 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:20.880805 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:20.881853 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "28e5691dcd034293b41ee1b92640e79d"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.887356 13183 heartbeater.cc:344] Connected to a master server at 127.12.104.254:33723
I20260924 15:08:20.887802 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260924 15:08:20.888051 13183 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:20.889096 13183 heartbeater.cc:507] Master 127.12.104.254:33723 requested a full tablet report, sending...
I20260924 15:08:20.891723 13055 ts_manager.cc:194] Registered new tserver with Master: 592c9b1c25db4f19aabd16f5640a961b (127.12.104.193:41297)
I20260924 15:08:20.891830 13196 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.892851 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 15:08:20.893211 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "28e5691dcd034293b41ee1b92640e79d"
format_stamp: "Formatted at 2026-09-24 15:08:20 on dist-test-slave-9164"
I20260924 15:08:20.893601 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:20.894239 13055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50232
I20260924 15:08:20.919890 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:20.920954 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:20.922433 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:20.925004 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:20.925200 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.925534 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:20.925683 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:20.971071 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:37185
I20260924 15:08:20.971199 13258 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:37185 every 8 connection(s)
I20260924 15:08:20.987515 13259 heartbeater.cc:344] Connected to a master server at 127.12.104.254:33723
I20260924 15:08:20.988195 13259 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:20.989064 13259 heartbeater.cc:507] Master 127.12.104.254:33723 requested a full tablet report, sending...
I20260924 15:08:20.992331 13055 ts_manager.cc:194] Registered new tserver with Master: 28e5691dcd034293b41ee1b92640e79d (127.12.104.194:37185)
I20260924 15:08:20.992462 12707 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.017756185s
I20260924 15:08:20.994438 13055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50238
I20260924 15:08:20.997578 13183 heartbeater.cc:499] Master 127.12.104.254:33723 was elected leader, sending a full tablet report...
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:422: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260924 15:08:21.018787 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:08:21.038592 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:21.056396 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:08:21.064877 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:08:21.083446 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:21.100548 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:08:21.108147 12707 master.cc:562] Master@127.12.104.254:33723 shutting down...
I20260924 15:08:21.125475 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:21.126370 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:21.127022 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 88f4142c5cac44589515bfb3afce03e1: stopping tablet replica
I20260924 15:08:21.148854 12707 master.cc:584] Master@127.12.104.254:33723 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (650 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260924 15:08:21.180563 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:34207
I20260924 15:08:21.181818 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:21.187322 13272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:21.188730 13273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:21.191568 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:21.191515 13275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:21.192934 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:21.193367 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:21.193609 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262501193572 us; error 0 us; skew 500 ppm
I20260924 15:08:21.194259 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:21.197191 12707 webserver.cc:533] Webserver started at http://127.12.104.254:35575/ using document root <none> and password file <none>
I20260924 15:08:21.197803 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:21.197988 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:21.198371 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:21.199559 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "cb1b32590e8a4ebfa762dba43bec91c2"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.205704 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260924 15:08:21.209870 13280 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.211055 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:08:21.211678 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "cb1b32590e8a4ebfa762dba43bec91c2"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.212061 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:21.236279 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:21.237504 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:21.282210 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:34207
I20260924 15:08:21.282436 13333 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:34207 every 8 connection(s)
I20260924 15:08:21.287489 13334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:21.298682 13334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2: Bootstrap starting.
I20260924 15:08:21.303184 13334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:21.307467 13334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2: No bootstrap required, opened a new log
I20260924 15:08:21.309405 13334 raft_consensus.cc:359] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER }
I20260924 15:08:21.309823 13334 raft_consensus.cc:385] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:21.310012 13334 raft_consensus.cc:740] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb1b32590e8a4ebfa762dba43bec91c2, State: Initialized, Role: FOLLOWER
I20260924 15:08:21.310617 13334 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER }
I20260924 15:08:21.311055 13334 raft_consensus.cc:399] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:08:21.311250 13334 raft_consensus.cc:493] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:08:21.311461 13334 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:21.316177 13334 raft_consensus.cc:515] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER }
I20260924 15:08:21.316710 13334 leader_election.cc:304] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb1b32590e8a4ebfa762dba43bec91c2; no voters:
I20260924 15:08:21.317988 13334 leader_election.cc:290] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:08:21.318432 13337 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:21.319971 13337 raft_consensus.cc:697] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 1 LEADER]: Becoming Leader. State: Replica: cb1b32590e8a4ebfa762dba43bec91c2, State: Running, Role: LEADER
I20260924 15:08:21.320797 13337 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER }
I20260924 15:08:21.321285 13334 sys_catalog.cc:565] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:21.323487 13339 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER } }
I20260924 15:08:21.324071 13339 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:21.324785 13340 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb1b32590e8a4ebfa762dba43bec91c2. Latest consensus state: current_term: 1 leader_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb1b32590e8a4ebfa762dba43bec91c2" member_type: VOTER } }
I20260924 15:08:21.326473 13340 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:21.331619 13343 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:21.336491 13343 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:21.339031 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:21.346421 13343 catalog_manager.cc:1396] Generated new cluster ID: b8800a0cab0747e9a24159feecd93374
I20260924 15:08:21.346779 13343 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:21.369274 13343 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:21.370729 13343 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:21.384723 13343 catalog_manager.cc:6219] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2: Generated new TSK 0
I20260924 15:08:21.385396 13343 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:21.408419 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:21.414980 13356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:21.415498 13357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:21.417164 13359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:21.417735 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:21.418581 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:21.418803 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:21.418980 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262501418961 us; error 0 us; skew 500 ppm
I20260924 15:08:21.419518 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:21.422151 12707 webserver.cc:533] Webserver started at http://127.12.104.193:34125/ using document root <none> and password file <none>
I20260924 15:08:21.422822 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:21.423029 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:21.423501 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:21.424726 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "57bacd77f92848b0b0daaf98fb4bc8af"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.430482 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 15:08:21.436051 13364 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.437218 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260924 15:08:21.437634 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "57bacd77f92848b0b0daaf98fb4bc8af"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.437973 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:21.450780 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:21.451987 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:21.453610 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:21.456806 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:21.457077 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.457397 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:21.457748 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.503986 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:43061
I20260924 15:08:21.504112 13427 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:43061 every 8 connection(s)
I20260924 15:08:21.509001 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:21.516004 13433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:21.517709 13435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:21.520605 13437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:21.520689 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:21.522014 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:21.522269 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:21.522518 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262501522486 us; error 0 us; skew 500 ppm
I20260924 15:08:21.523272 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:21.523537 13428 heartbeater.cc:344] Connected to a master server at 127.12.104.254:34207
I20260924 15:08:21.523939 13428 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:21.524657 13428 heartbeater.cc:507] Master 127.12.104.254:34207 requested a full tablet report, sending...
I20260924 15:08:21.526382 12707 webserver.cc:533] Webserver started at http://127.12.104.194:39243/ using document root <none> and password file <none>
I20260924 15:08:21.526772 13299 ts_manager.cc:194] Registered new tserver with Master: 57bacd77f92848b0b0daaf98fb4bc8af (127.12.104.193:43061)
I20260924 15:08:21.526953 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:21.527261 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:21.527547 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:21.528661 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.528684 13299 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55734
I20260924 15:08:21.533502 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260924 15:08:21.537184 13442 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.538250 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 15:08:21.538715 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654"
format_stamp: "Formatted at 2026-09-24 15:08:21 on dist-test-slave-9164"
I20260924 15:08:21.538990 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:21.562347 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:21.563365 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:21.564823 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:21.567113 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:21.567301 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.567618 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:21.567762 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:21.609606 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:39101
I20260924 15:08:21.609730 13506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:39101 every 8 connection(s)
I20260924 15:08:21.626243 13507 heartbeater.cc:344] Connected to a master server at 127.12.104.254:34207
I20260924 15:08:21.626837 13507 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:21.627579 13507 heartbeater.cc:507] Master 127.12.104.254:34207 requested a full tablet report, sending...
I20260924 15:08:21.629877 13299 ts_manager.cc:194] Registered new tserver with Master: 6fcd7e4e1f29404f9c8fbe8ec974b654 (127.12.104.194:39101)
I20260924 15:08:21.630455 12707 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.017403826s
I20260924 15:08:21.631340 13428 heartbeater.cc:499] Master 127.12.104.254:34207 was elected leader, sending a full tablet report...
I20260924 15:08:21.631893 13299 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55742
I20260924 15:08:21.656582 13299 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 15:08:21.658167 13299 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260924 15:08:21.704545 13472 tablet_service.cc:1511] Processing CreateTablet for tablet cfcb3ae49fc84733b21f9f01b0df8cb1 (DEFAULT_TABLE table=test-table [id=4983993006104f51b55d98be4a327621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:21.705751 13472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfcb3ae49fc84733b21f9f01b0df8cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:21.706568 13392 tablet_service.cc:1511] Processing CreateTablet for tablet cfcb3ae49fc84733b21f9f01b0df8cb1 (DEFAULT_TABLE table=test-table [id=4983993006104f51b55d98be4a327621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:21.707710 13392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfcb3ae49fc84733b21f9f01b0df8cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:21.718982 13520 tablet_bootstrap.cc:492] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: Bootstrap starting.
I20260924 15:08:21.720409 13521 tablet_bootstrap.cc:492] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: Bootstrap starting.
I20260924 15:08:21.724833 13520 tablet_bootstrap.cc:654] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:21.726473 13521 tablet_bootstrap.cc:654] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:21.729723 13520 tablet_bootstrap.cc:492] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: No bootstrap required, opened a new log
I20260924 15:08:21.730129 13520 ts_tablet_manager.cc:1403] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260924 15:08:21.730822 13521 tablet_bootstrap.cc:492] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: No bootstrap required, opened a new log
I20260924 15:08:21.731211 13521 ts_tablet_manager.cc:1403] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260924 15:08:21.732231 13520 raft_consensus.cc:359] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:21.732776 13520 raft_consensus.cc:740] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcd7e4e1f29404f9c8fbe8ec974b654, State: Initialized, Role: FOLLOWER
I20260924 15:08:21.733300 13521 raft_consensus.cc:359] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:21.733363 13520 consensus_queue.cc:260] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:21.733933 13521 raft_consensus.cc:740] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Initialized, Role: FOLLOWER
I20260924 15:08:21.734985 13507 heartbeater.cc:499] Master 127.12.104.254:34207 was elected leader, sending a full tablet report...
I20260924 15:08:21.734622 13521 consensus_queue.cc:260] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:21.738489 13521 ts_tablet_manager.cc:1434] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260924 15:08:21.741343 13520 ts_tablet_manager.cc:1434] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260924 15:08:21.785086 12707 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260924 15:08:21.785339 12707 raft_consensus.cc:423] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 0 FOLLOWER]: Emulating election...
I20260924 15:08:21.785530 12707 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:21.790207 12707 raft_consensus.cc:697] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 1 LEADER]: Becoming Leader. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Running, Role: LEADER
W20260924 15:08:21.790774 13428 heartbeater.cc:646] Failed to heartbeat to 127.12.104.254:34207 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260924 15:08:21.791091 12707 consensus_queue.cc:237] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:21.794250 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:21.816339 12707 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0
I20260924 15:08:21.921072 12707 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0
I20260924 15:08:22.127673 12707 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0
I20260924 15:08:22.421456 13482 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:22.428057 13528 consensus_queue.cc:1048] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:22.442085 12707 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260924 15:08:22.442430 12707 raft_consensus.cc:423] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 1 LEADER]: Emulating election...
I20260924 15:08:22.442752 12707 raft_consensus.cc:3055] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 1 LEADER]: Stepping down as leader of term 1
I20260924 15:08:22.443017 12707 raft_consensus.cc:740] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Running, Role: LEADER
I20260924 15:08:22.443543 12707 consensus_queue.cc:260] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:22.444249 12707 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:22.448591 12707 raft_consensus.cc:697] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 2 LEADER]: Becoming Leader. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Running, Role: LEADER
I20260924 15:08:22.449270 12707 consensus_queue.cc:237] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:22.450759 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:22.456157 12707 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260924 15:08:22.562645 12707 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260924 15:08:22.768263 12707 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260924 15:08:23.076745 13481 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:23.077765 12707 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260924 15:08:23.085446 13481 raft_consensus.cc:1275] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 2 FOLLOWER]: Refusing update from remote peer 57bacd77f92848b0b0daaf98fb4bc8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 15:08:23.087281 13537 consensus_queue.cc:1048] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 15:08:23.486150 12707 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260924 15:08:23.486487 12707 raft_consensus.cc:423] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 2 LEADER]: Emulating election...
I20260924 15:08:23.486832 12707 raft_consensus.cc:3055] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 2 LEADER]: Stepping down as leader of term 2
I20260924 15:08:23.487087 12707 raft_consensus.cc:740] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Running, Role: LEADER
I20260924 15:08:23.487663 12707 consensus_queue.cc:260] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:23.488492 12707 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:23.493984 12707 raft_consensus.cc:697] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 3 LEADER]: Becoming Leader. State: Replica: 57bacd77f92848b0b0daaf98fb4bc8af, State: Running, Role: LEADER
I20260924 15:08:23.494793 12707 consensus_queue.cc:237] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57bacd77f92848b0b0daaf98fb4bc8af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43061 } } peers { permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 } }
I20260924 15:08:23.496476 12707 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260924 15:08:23.501312 12707 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260924 15:08:23.606793 12707 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260924 15:08:23.812278 12707 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260924 15:08:23.921252 13481 raft_consensus.cc:3060] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:23.926748 13481 raft_consensus.cc:1275] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 3 FOLLOWER]: Refusing update from remote peer 57bacd77f92848b0b0daaf98fb4bc8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 15:08:23.928396 13537 consensus_queue.cc:1048] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcd7e4e1f29404f9c8fbe8ec974b654" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 15:08:24.125129 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:08:24.157210 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:24.158113 12707 tablet_replica.cc:333] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af: stopping tablet replica
I20260924 15:08:24.158767 12707 raft_consensus.cc:2243] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 3 LEADER]: Raft consensus shutting down.
I20260924 15:08:24.159438 12707 raft_consensus.cc:2272] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 57bacd77f92848b0b0daaf98fb4bc8af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:24.180794 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:08:24.193073 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:08:24.218364 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:24.219739 12707 tablet_replica.cc:333] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654: stopping tablet replica
I20260924 15:08:24.220503 12707 raft_consensus.cc:2243] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 15:08:24.221100 12707 raft_consensus.cc:2272] T cfcb3ae49fc84733b21f9f01b0df8cb1 P 6fcd7e4e1f29404f9c8fbe8ec974b654 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:24.243970 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:08:24.255532 12707 master.cc:562] Master@127.12.104.254:34207 shutting down...
I20260924 15:08:24.281239 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:24.281858 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:24.282166 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P cb1b32590e8a4ebfa762dba43bec91c2: stopping tablet replica
I20260924 15:08:24.303200 12707 master.cc:584] Master@127.12.104.254:34207 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (3159 ms)
[----------] 3 tests from LeadershipChangeReportingTest (11307 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260924 15:08:24.339926 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:45611
I20260924 15:08:24.341323 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:24.347891 13551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.348347 13553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.350142 13555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:24.351455 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:24.352427 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:24.352633 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:24.352759 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262504352747 us; error 0 us; skew 500 ppm
I20260924 15:08:24.353260 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:24.356916 12707 webserver.cc:533] Webserver started at http://127.12.104.254:33551/ using document root <none> and password file <none>
I20260924 15:08:24.357539 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:24.357725 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:24.357996 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:24.360121 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "814dc075977b4b5084991ec8aa8466fe"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.368717 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.003s
I20260924 15:08:24.373948 13560 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.374920 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 15:08:24.375269 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "814dc075977b4b5084991ec8aa8466fe"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.375592 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:24.387874 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:24.389261 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:24.435247 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:45611
I20260924 15:08:24.435374 13611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:45611 every 8 connection(s)
I20260924 15:08:24.439620 13612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:24.450814 13612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe: Bootstrap starting.
I20260924 15:08:24.456382 13612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:24.461835 13612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe: No bootstrap required, opened a new log
I20260924 15:08:24.464574 13612 raft_consensus.cc:359] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER }
I20260924 15:08:24.465065 13612 raft_consensus.cc:385] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:24.465581 13612 raft_consensus.cc:740] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814dc075977b4b5084991ec8aa8466fe, State: Initialized, Role: FOLLOWER
I20260924 15:08:24.466336 13612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER }
I20260924 15:08:24.467155 13612 raft_consensus.cc:399] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:08:24.467492 13612 raft_consensus.cc:493] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:08:24.467775 13612 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:24.472994 13612 raft_consensus.cc:515] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER }
I20260924 15:08:24.473615 13612 leader_election.cc:304] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 814dc075977b4b5084991ec8aa8466fe; no voters:
I20260924 15:08:24.475025 13612 leader_election.cc:290] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:08:24.475504 13615 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:24.477069 13615 raft_consensus.cc:697] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 1 LEADER]: Becoming Leader. State: Replica: 814dc075977b4b5084991ec8aa8466fe, State: Running, Role: LEADER
I20260924 15:08:24.477900 13615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER }
I20260924 15:08:24.478490 13612 sys_catalog.cc:565] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:24.483007 13617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 814dc075977b4b5084991ec8aa8466fe. Latest consensus state: current_term: 1 leader_uuid: "814dc075977b4b5084991ec8aa8466fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER } }
I20260924 15:08:24.483688 13617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:24.484031 13616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "814dc075977b4b5084991ec8aa8466fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814dc075977b4b5084991ec8aa8466fe" member_type: VOTER } }
I20260924 15:08:24.484804 13616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:24.487596 13621 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:24.496690 13621 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:24.498214 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:24.506538 13621 catalog_manager.cc:1396] Generated new cluster ID: 0edfac58014142f982d092c1a2dfe12b
I20260924 15:08:24.506862 13621 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:24.529843 13621 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:24.531524 13621 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:24.550740 13621 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe: Generated new TSK 0
I20260924 15:08:24.551433 13621 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:24.565688 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:24.574178 13633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.575304 13634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.577224 13636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:24.578166 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:24.579071 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:24.579267 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:24.579460 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262504579443 us; error 0 us; skew 500 ppm
I20260924 15:08:24.580286 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:24.583102 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40715/ using document root <none> and password file <none>
I20260924 15:08:24.583659 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:24.583894 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:24.584203 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:24.585456 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "a950830879c24f86981e4b0dd63fec7f"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.590879 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 15:08:24.595814 13641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.596874 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260924 15:08:24.597164 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "a950830879c24f86981e4b0dd63fec7f"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.597460 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:24.619163 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:24.620411 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:24.622001 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:24.624634 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:24.624854 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.625190 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:24.625383 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.678112 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:43301
I20260924 15:08:24.678272 13705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:43301 every 8 connection(s)
I20260924 15:08:24.683769 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:24.691223 13710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.692595 13711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:24.697118 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:24.697387 13713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:24.698784 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:24.699133 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:24.699443 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262504699419 us; error 0 us; skew 500 ppm
I20260924 15:08:24.700567 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:24.700681 13706 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45611
I20260924 15:08:24.701335 13706 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:24.702720 13706 heartbeater.cc:507] Master 127.12.104.254:45611 requested a full tablet report, sending...
I20260924 15:08:24.704171 12707 webserver.cc:533] Webserver started at http://127.12.104.194:44595/ using document root <none> and password file <none>
I20260924 15:08:24.704862 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:24.705114 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:24.705830 13577 ts_manager.cc:194] Registered new tserver with Master: a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:24.705967 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:24.707779 13577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52788
I20260924 15:08:24.707872 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "c31e55c432d541d886cbe5dec62ee982"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.714164 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 15:08:24.718451 13718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.719465 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 15:08:24.719741 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "c31e55c432d541d886cbe5dec62ee982"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.720000 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:24.738212 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:24.739512 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:24.740994 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:24.743420 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:24.743623 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.743924 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:24.744076 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.793104 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:35837
I20260924 15:08:24.793212 13782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:35837 every 8 connection(s)
I20260924 15:08:24.798460 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:24.805416 13786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.806730 13787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:24.810590 13789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:24.811205 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:24.812422 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:24.812708 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:24.812917 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262504812879 us; error 0 us; skew 500 ppm
I20260924 15:08:24.813225 13783 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45611
I20260924 15:08:24.813512 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:24.813589 13783 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:24.814610 13783 heartbeater.cc:507] Master 127.12.104.254:45611 requested a full tablet report, sending...
I20260924 15:08:24.816597 12707 webserver.cc:533] Webserver started at http://127.12.104.195:33469/ using document root <none> and password file <none>
I20260924 15:08:24.817088 13577 ts_manager.cc:194] Registered new tserver with Master: c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837)
I20260924 15:08:24.817310 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:24.817597 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:24.817953 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:24.819029 13577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52802
I20260924 15:08:24.819705 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-2-root/instance:
uuid: "de41e61425f242958c49dc01987e081c"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.825436 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 15:08:24.829849 13794 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.830771 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:08:24.831131 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-2-root
uuid: "de41e61425f242958c49dc01987e081c"
format_stamp: "Formatted at 2026-09-24 15:08:24 on dist-test-slave-9164"
I20260924 15:08:24.831509 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790262492846473-12707-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:24.867444 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:24.868851 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:24.870673 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:24.873409 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:24.873692 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.873973 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:24.874176 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:24.956423 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.195:39463
I20260924 15:08:24.956524 13858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.195:39463 every 8 connection(s)
I20260924 15:08:24.974900 13859 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45611
I20260924 15:08:24.975323 13859 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:24.976158 13859 heartbeater.cc:507] Master 127.12.104.254:45611 requested a full tablet report, sending...
I20260924 15:08:24.978928 13577 ts_manager.cc:194] Registered new tserver with Master: de41e61425f242958c49dc01987e081c (127.12.104.195:39463)
I20260924 15:08:24.979429 12707 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017897914s
I20260924 15:08:24.980721 13577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52804
I20260924 15:08:25.008477 13577 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260924 15:08:25.010143 13577 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 15:08:25.054033 13747 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7 (DEFAULT_TABLE table=test-table [id=ddf46b55bfe64a57bb17a89c1c8cc425]), partition=
I20260924 15:08:25.055437 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:25.062578 13823 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7 (DEFAULT_TABLE table=test-table [id=ddf46b55bfe64a57bb17a89c1c8cc425]), partition=
I20260924 15:08:25.064142 13823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:25.067438 13669 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7 (DEFAULT_TABLE table=test-table [id=ddf46b55bfe64a57bb17a89c1c8cc425]), partition=
I20260924 15:08:25.068729 13669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca85fbe8ac74dc3b34bfdf62f3963d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:25.074905 13874 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: Bootstrap starting.
I20260924 15:08:25.083091 13875 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: Bootstrap starting.
I20260924 15:08:25.083835 13874 tablet_bootstrap.cc:654] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:25.091526 13874 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: No bootstrap required, opened a new log
I20260924 15:08:25.092087 13877 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: Bootstrap starting.
I20260924 15:08:25.092832 13875 tablet_bootstrap.cc:654] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:25.092072 13874 ts_tablet_manager.cc:1403] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20260924 15:08:25.097849 13874 raft_consensus.cc:359] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.098663 13874 raft_consensus.cc:385] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:25.098968 13874 raft_consensus.cc:740] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c31e55c432d541d886cbe5dec62ee982, State: Initialized, Role: FOLLOWER
I20260924 15:08:25.100023 13874 consensus_queue.cc:260] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.101015 13875 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: No bootstrap required, opened a new log
I20260924 15:08:25.101130 13877 tablet_bootstrap.cc:654] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:25.101651 13875 ts_tablet_manager.cc:1403] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.010s
I20260924 15:08:25.103875 13783 heartbeater.cc:499] Master 127.12.104.254:45611 was elected leader, sending a full tablet report...
I20260924 15:08:25.105031 13874 ts_tablet_manager.cc:1434] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260924 15:08:25.105644 13875 raft_consensus.cc:359] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.106407 13875 raft_consensus.cc:385] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:25.106829 13875 raft_consensus.cc:740] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de41e61425f242958c49dc01987e081c, State: Initialized, Role: FOLLOWER
I20260924 15:08:25.107797 13875 consensus_queue.cc:260] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.111958 13859 heartbeater.cc:499] Master 127.12.104.254:45611 was elected leader, sending a full tablet report...
I20260924 15:08:25.112546 13875 ts_tablet_manager.cc:1434] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260924 15:08:25.114837 13877 tablet_bootstrap.cc:492] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: No bootstrap required, opened a new log
I20260924 15:08:25.115396 13877 ts_tablet_manager.cc:1403] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.007s
I20260924 15:08:25.118134 13877 raft_consensus.cc:359] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.118961 13877 raft_consensus.cc:385] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:25.119264 13877 raft_consensus.cc:740] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a950830879c24f86981e4b0dd63fec7f, State: Initialized, Role: FOLLOWER
I20260924 15:08:25.120031 13877 consensus_queue.cc:260] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.123935 13706 heartbeater.cc:499] Master 127.12.104.254:45611 was elected leader, sending a full tablet report...
I20260924 15:08:25.126188 13877 ts_tablet_manager.cc:1434] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260924 15:08:25.365069 13882 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:25.365662 13882 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.368144 13882 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), de41e61425f242958c49dc01987e081c (127.12.104.195:39463)
I20260924 15:08:25.380522 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "a950830879c24f86981e4b0dd63fec7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982" is_pre_election: true
I20260924 15:08:25.381217 13834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "a950830879c24f86981e4b0dd63fec7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de41e61425f242958c49dc01987e081c" is_pre_election: true
I20260924 15:08:25.381578 13757 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a950830879c24f86981e4b0dd63fec7f in term 0.
I20260924 15:08:25.382174 13834 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a950830879c24f86981e4b0dd63fec7f in term 0.
I20260924 15:08:25.383466 13644 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a950830879c24f86981e4b0dd63fec7f, c31e55c432d541d886cbe5dec62ee982; no voters:
I20260924 15:08:25.384634 13882 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:25.385115 13882 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:25.385452 13882 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:25.391000 13882 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.392988 13882 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [CANDIDATE]: Term 1 election: Requested vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), de41e61425f242958c49dc01987e081c (127.12.104.195:39463)
I20260924 15:08:25.394112 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "a950830879c24f86981e4b0dd63fec7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982"
I20260924 15:08:25.394188 13834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "a950830879c24f86981e4b0dd63fec7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de41e61425f242958c49dc01987e081c"
I20260924 15:08:25.394831 13757 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:25.394932 13834 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:25.400780 13757 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a950830879c24f86981e4b0dd63fec7f in term 1.
I20260924 15:08:25.400785 13834 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a950830879c24f86981e4b0dd63fec7f in term 1.
I20260924 15:08:25.402145 13644 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a950830879c24f86981e4b0dd63fec7f, c31e55c432d541d886cbe5dec62ee982; no voters:
I20260924 15:08:25.402976 13882 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:25.403973 13882 raft_consensus.cc:697] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 1 LEADER]: Becoming Leader. State: Replica: a950830879c24f86981e4b0dd63fec7f, State: Running, Role: LEADER
I20260924 15:08:25.404790 13882 consensus_queue.cc:237] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:25.413533 13577 catalog_manager.cc:5838] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f reported cstate change: term changed from 0 to 1, leader changed from <none> to a950830879c24f86981e4b0dd63fec7f (127.12.104.193). New cstate: current_term: 1 leader_uuid: "a950830879c24f86981e4b0dd63fec7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:25.855808 13882 consensus_queue.cc:1048] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:25.857122 13885 consensus_queue.cc:1048] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:25.991276 12707 tablet_replica.cc:333] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f: stopping tablet replica
I20260924 15:08:25.992017 12707 raft_consensus.cc:2243] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:25.993166 12707 raft_consensus.cc:2272] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P a950830879c24f86981e4b0dd63fec7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:27.374053 13896 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a950830879c24f86981e4b0dd63fec7f)
I20260924 15:08:27.374157 13898 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a950830879c24f86981e4b0dd63fec7f)
I20260924 15:08:27.374835 13896 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:27.374876 13898 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:27.383864 13896 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de41e61425f242958c49dc01987e081c (127.12.104.195:39463), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:27.389966 13898 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:27.405637 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "c31e55c432d541d886cbe5dec62ee982" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f" is_pre_election: true
W20260924 15:08:27.407912 13719 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:27.409065 13834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "c31e55c432d541d886cbe5dec62ee982" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de41e61425f242958c49dc01987e081c" is_pre_election: true
I20260924 15:08:27.409344 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f" is_pre_election: true
I20260924 15:08:27.409758 13834 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c31e55c432d541d886cbe5dec62ee982 in term 1.
I20260924 15:08:27.409991 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982" is_pre_election: true
W20260924 15:08:27.410748 13795 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:27.410820 13757 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de41e61425f242958c49dc01987e081c in term 1.
I20260924 15:08:27.411374 13720 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c31e55c432d541d886cbe5dec62ee982, de41e61425f242958c49dc01987e081c; no voters: a950830879c24f86981e4b0dd63fec7f
I20260924 15:08:27.412057 13896 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:08:27.412395 13896 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a950830879c24f86981e4b0dd63fec7f)
I20260924 15:08:27.412827 13896 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:27.412781 13797 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c31e55c432d541d886cbe5dec62ee982, de41e61425f242958c49dc01987e081c; no voters: a950830879c24f86981e4b0dd63fec7f
I20260924 15:08:27.413750 13898 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:08:27.414090 13898 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Starting leader election (detected failure of leader a950830879c24f86981e4b0dd63fec7f)
I20260924 15:08:27.414434 13898 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:27.418780 13896 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:27.420030 13898 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:27.420885 13896 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 election: Requested vote from peers de41e61425f242958c49dc01987e081c (127.12.104.195:39463), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:27.421911 13834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "c31e55c432d541d886cbe5dec62ee982" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de41e61425f242958c49dc01987e081c"
I20260924 15:08:27.422282 13898 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 election: Requested vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:27.422361 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "c31e55c432d541d886cbe5dec62ee982" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f"
I20260924 15:08:27.422901 13834 raft_consensus.cc:2393] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c31e55c432d541d886cbe5dec62ee982 in current term 2: Already voted for candidate de41e61425f242958c49dc01987e081c in this term.
W20260924 15:08:27.423909 13719 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:27.423866 13679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f"
I20260924 15:08:27.424871 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982"
I20260924 15:08:27.424917 13720 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c31e55c432d541d886cbe5dec62ee982; no voters: a950830879c24f86981e4b0dd63fec7f, de41e61425f242958c49dc01987e081c
W20260924 15:08:27.425374 13795 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:27.425909 13757 raft_consensus.cc:2393] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate de41e61425f242958c49dc01987e081c in current term 2: Already voted for candidate c31e55c432d541d886cbe5dec62ee982 in this term.
I20260924 15:08:27.426380 13896 raft_consensus.cc:2749] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 15:08:27.427265 13797 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de41e61425f242958c49dc01987e081c; no voters: a950830879c24f86981e4b0dd63fec7f, c31e55c432d541d886cbe5dec62ee982
I20260924 15:08:27.427917 13898 raft_consensus.cc:2749] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 15:08:29.332008 13907 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:29.332535 13907 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:29.334411 13907 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:29.335594 13679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f" is_pre_election: true
I20260924 15:08:29.335711 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982" is_pre_election: true
I20260924 15:08:29.336347 13757 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de41e61425f242958c49dc01987e081c in term 2.
W20260924 15:08:29.337014 13795 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:29.337554 13797 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c31e55c432d541d886cbe5dec62ee982, de41e61425f242958c49dc01987e081c; no voters: a950830879c24f86981e4b0dd63fec7f
I20260924 15:08:29.338569 13907 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 15:08:29.338995 13907 raft_consensus.cc:493] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:29.339466 13907 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:29.344749 13907 raft_consensus.cc:515] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:29.346619 13907 leader_election.cc:290] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 election: Requested vote from peers c31e55c432d541d886cbe5dec62ee982 (127.12.104.194:35837), a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301)
I20260924 15:08:29.347734 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c31e55c432d541d886cbe5dec62ee982"
I20260924 15:08:29.347909 13679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca85fbe8ac74dc3b34bfdf62f3963d7" candidate_uuid: "de41e61425f242958c49dc01987e081c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a950830879c24f86981e4b0dd63fec7f"
I20260924 15:08:29.348702 13757 raft_consensus.cc:3060] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 2 FOLLOWER]: Advancing to term 3
W20260924 15:08:29.349498 13795 leader_election.cc:343] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Illegal state: cannot vote while shut down
I20260924 15:08:29.353839 13757 raft_consensus.cc:2468] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de41e61425f242958c49dc01987e081c in term 3.
I20260924 15:08:29.355008 13797 leader_election.cc:304] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c31e55c432d541d886cbe5dec62ee982, de41e61425f242958c49dc01987e081c; no voters: a950830879c24f86981e4b0dd63fec7f
I20260924 15:08:29.355983 13907 raft_consensus.cc:2804] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 3 FOLLOWER]: Leader election won for term 3
I20260924 15:08:29.357362 13907 raft_consensus.cc:697] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 3 LEADER]: Becoming Leader. State: Replica: de41e61425f242958c49dc01987e081c, State: Running, Role: LEADER
I20260924 15:08:29.358423 13907 consensus_queue.cc:237] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 } } peers { permanent_uuid: "de41e61425f242958c49dc01987e081c" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39463 } } peers { permanent_uuid: "a950830879c24f86981e4b0dd63fec7f" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43301 } }
I20260924 15:08:29.818050 13795 consensus_queue.cc:579] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [LEADER]: The tablet replica hosted on peer a950830879c24f86981e4b0dd63fec7f has failed
W20260924 15:08:29.818728 13795 consensus_peers.cc:597] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c -> Peer a950830879c24f86981e4b0dd63fec7f (127.12.104.193:43301): Couldn't send request to peer a950830879c24f86981e4b0dd63fec7f. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:08:29.898761 13757 raft_consensus.cc:1275] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 3 FOLLOWER]: Refusing update from remote peer de41e61425f242958c49dc01987e081c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 15:08:29.901185 13907 consensus_queue.cc:1048] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c31e55c432d541d886cbe5dec62ee982" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 15:08:30.033903 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:08:30.056308 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:30.076490 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:08:30.086787 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:08:30.106554 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:30.107529 12707 tablet_replica.cc:333] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982: stopping tablet replica
I20260924 15:08:30.108225 12707 raft_consensus.cc:2243] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 15:08:30.108784 12707 raft_consensus.cc:2272] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P c31e55c432d541d886cbe5dec62ee982 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:30.130615 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:08:30.140651 12707 tablet_server.cc:179] TabletServer@127.12.104.195:0 shutting down...
I20260924 15:08:30.159457 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:30.160131 12707 tablet_replica.cc:333] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c: stopping tablet replica
I20260924 15:08:30.160668 12707 raft_consensus.cc:2243] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 3 LEADER]: Raft consensus shutting down.
I20260924 15:08:30.161464 12707 raft_consensus.cc:2272] T 7ca85fbe8ac74dc3b34bfdf62f3963d7 P de41e61425f242958c49dc01987e081c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:30.171819 12707 tablet_server.cc:196] TabletServer@127.12.104.195:0 shutdown complete.
I20260924 15:08:30.180838 12707 master.cc:562] Master@127.12.104.254:45611 shutting down...
I20260924 15:08:30.202415 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:30.203042 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:30.203325 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 814dc075977b4b5084991ec8aa8466fe: stopping tablet replica
I20260924 15:08:30.221315 12707 master.cc:584] Master@127.12.104.254:45611 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (5910 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260924 15:08:30.251926 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:41463
I20260924 15:08:30.253062 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:30.258915 13918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:30.260476 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:30.262686 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.263274 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:30.264052 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:30.264259 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:30.264449 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262510264433 us; error 0 us; skew 500 ppm
I20260924 15:08:30.265103 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:30.267858 12707 webserver.cc:533] Webserver started at http://127.12.104.254:45001/ using document root <none> and password file <none>
I20260924 15:08:30.268404 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:30.268605 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:30.268914 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:30.270138 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "c66cbed0d93b4abb84b655302f986557"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.275377 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 15:08:30.279585 13926 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.280706 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 15:08:30.281001 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "c66cbed0d93b4abb84b655302f986557"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.281308 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:30.299420 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:30.300546 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:30.342532 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:41463
I20260924 15:08:30.342648 13978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:41463 every 8 connection(s)
I20260924 15:08:30.346735 13979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:30.359622 13979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557: Bootstrap starting.
I20260924 15:08:30.368449 13979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:30.375993 13979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557: No bootstrap required, opened a new log
I20260924 15:08:30.379187 13979 raft_consensus.cc:359] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER }
I20260924 15:08:30.379854 13979 raft_consensus.cc:385] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:30.380198 13979 raft_consensus.cc:740] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c66cbed0d93b4abb84b655302f986557, State: Initialized, Role: FOLLOWER
I20260924 15:08:30.381179 13979 consensus_queue.cc:260] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER }
I20260924 15:08:30.382085 13979 raft_consensus.cc:399] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:08:30.382441 13979 raft_consensus.cc:493] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:08:30.382833 13979 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:30.391513 13979 raft_consensus.cc:515] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER }
I20260924 15:08:30.392205 13979 leader_election.cc:304] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c66cbed0d93b4abb84b655302f986557; no voters:
I20260924 15:08:30.393963 13979 leader_election.cc:290] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:08:30.394246 13982 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:30.396075 13982 raft_consensus.cc:697] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 1 LEADER]: Becoming Leader. State: Replica: c66cbed0d93b4abb84b655302f986557, State: Running, Role: LEADER
I20260924 15:08:30.397357 13979 sys_catalog.cc:565] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:30.397009 13982 consensus_queue.cc:237] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER }
I20260924 15:08:30.400674 13983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c66cbed0d93b4abb84b655302f986557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER } }
I20260924 15:08:30.402065 13984 sys_catalog.cc:455] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c66cbed0d93b4abb84b655302f986557. Latest consensus state: current_term: 1 leader_uuid: "c66cbed0d93b4abb84b655302f986557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c66cbed0d93b4abb84b655302f986557" member_type: VOTER } }
I20260924 15:08:30.402509 13983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:30.402889 13984 sys_catalog.cc:458] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:30.406024 13990 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:30.414851 13990 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:30.415899 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:30.424827 13990 catalog_manager.cc:1396] Generated new cluster ID: 4f67902cd6fd4a958e4816fe85661550
I20260924 15:08:30.425170 13990 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:30.449205 13990 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:30.451305 13990 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:30.466574 13990 catalog_manager.cc:6219] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557: Generated new TSK 0
I20260924 15:08:30.467223 13990 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:30.483255 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:30.489897 14000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:30.490973 14001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.493468 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:30.493602 14003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.494845 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:30.495069 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:30.495297 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262510495271 us; error 0 us; skew 500 ppm
I20260924 15:08:30.496027 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:30.499372 12707 webserver.cc:533] Webserver started at http://127.12.104.193:36201/ using document root <none> and password file <none>
I20260924 15:08:30.500540 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:30.500797 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:30.501205 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:30.502640 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "84425e07ba594549b387ee2750d0664d"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.508280 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 15:08:30.513223 14008 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.514739 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 15:08:30.515178 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "84425e07ba594549b387ee2750d0664d"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.515766 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:30.542526 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:30.543783 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:30.545356 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:30.547868 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:30.548128 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.548401 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:30.548667 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.596252 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:45837
I20260924 15:08:30.596395 14073 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:45837 every 8 connection(s)
I20260924 15:08:30.614200 14074 heartbeater.cc:344] Connected to a master server at 127.12.104.254:41463
I20260924 15:08:30.614641 14074 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:30.615734 14074 heartbeater.cc:507] Master 127.12.104.254:41463 requested a full tablet report, sending...
I20260924 15:08:30.618852 13943 ts_manager.cc:194] Registered new tserver with Master: 84425e07ba594549b387ee2750d0664d (127.12.104.193:45837)
I20260924 15:08:30.619743 12707 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.019652283s
I20260924 15:08:30.620788 13943 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58784
I20260924 15:08:30.642974 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:08:30.659494 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:30.677312 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:08:30.686856 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260924 15:08:30.689997 12707 tablet_server.cc:94] deduplicated master addresses: 127.12.104.254:41463
W20260924 15:08:30.693104 14084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:30.693979 14085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.696260 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:30.696516 14087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.697502 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:30.697717 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:30.697933 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262510697896 us; error 0 us; skew 500 ppm
I20260924 15:08:30.698670 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:30.701467 12707 webserver.cc:533] Webserver started at http://127.12.104.193:36201/ using document root <none> and password file <none>
I20260924 15:08:30.702055 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:30.702271 12707 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260924 15:08:30.706768 12707 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 15:08:30.710227 14092 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.711354 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 15:08:30.711625 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "84425e07ba594549b387ee2750d0664d"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.711905 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:30.738639 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:30.739665 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:30.741137 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:30.744274 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:30.744534 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.744897 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:30.745065 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.793218 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:45837
I20260924 15:08:30.793315 14155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:45837 every 8 connection(s)
I20260924 15:08:30.800313 12707 tablet_server.cc:179] TabletServer@127.12.104.193:45837 shutting down...
I20260924 15:08:30.823776 14156 heartbeater.cc:344] Connected to a master server at 127.12.104.254:41463
I20260924 15:08:30.824750 14156 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:30.826192 14156 heartbeater.cc:507] Master 127.12.104.254:41463 requested a full tablet report, sending...
I20260924 15:08:30.829157 13943 ts_manager.cc:194] Re-registered known tserver with Master: 84425e07ba594549b387ee2750d0664d (127.12.104.193:45837)
I20260924 15:08:30.830642 13943 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58806
I20260924 15:08:30.833619 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:08:30.850502 12707 tablet_server.cc:196] TabletServer@127.12.104.193:45837 shutdown complete.
I20260924 15:08:30.860569 12707 master.cc:562] Master@127.12.104.254:41463 shutting down...
I20260924 15:08:30.880842 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:08:30.881625 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:08:30.882228 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P c66cbed0d93b4abb84b655302f986557: stopping tablet replica
I20260924 15:08:30.907549 12707 master.cc:584] Master@127.12.104.254:41463 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (683 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260924 15:08:30.933552 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:45779,127.12.104.253:39543,127.12.104.252:41327
I20260924 15:08:30.935153 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:30.941039 14163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:30.942164 14164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.944568 12707 server_base.cc:1061] running on GCE node
W20260924 15:08:30.944494 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:30.945770 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:30.946007 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:30.946193 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262510946165 us; error 0 us; skew 500 ppm
I20260924 15:08:30.946779 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:30.949415 12707 webserver.cc:533] Webserver started at http://127.12.104.254:37671/ using document root <none> and password file <none>
I20260924 15:08:30.949978 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:30.950210 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:30.950541 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:30.951771 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "08536d2250d14b76957e5b87dd30a9e7"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.957923 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260924 15:08:30.962997 14171 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:30.964200 12707 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260924 15:08:30.964565 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "08536d2250d14b76957e5b87dd30a9e7"
format_stamp: "Formatted at 2026-09-24 15:08:30 on dist-test-slave-9164"
I20260924 15:08:30.964920 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:30.978159 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:30.979364 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.022079 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:45779
I20260924 15:08:31.022221 14224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:45779 every 8 connection(s)
I20260924 15:08:31.026664 14225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:31.026714 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:31.032656 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:31.035036 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.036334 14225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } has no permanent_uuid. Determining permanent_uuid...
W20260924 15:08:31.036617 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.036688 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:31.037938 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:31.038203 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:31.038451 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262511038428 us; error 0 us; skew 500 ppm
I20260924 15:08:31.039068 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:31.048332 12707 webserver.cc:533] Webserver started at http://127.12.104.253:43889/ using document root <none> and password file <none>
I20260924 15:08:31.049078 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:31.049350 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:31.049708 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:31.050987 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-1-root/instance:
uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.056926 14225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.057778 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
W20260924 15:08:31.061071 14172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.104.253:39543: connect: Connection refused (error 111)
I20260924 15:08:31.063912 14238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.065111 12707 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260924 15:08:31.065474 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-1-root
uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
W20260924 15:08:31.065855 14225 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.12.104.253:39543: Network error: Client connection negotiation failed: client connection to 127.12.104.253:39543: connect: Connection refused (error 111)
I20260924 15:08:31.065935 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:31.080371 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:31.081521 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.090619 14225 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } attempt: 1
W20260924 15:08:31.097143 14225 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.12.104.253:39543: Network error: Client connection negotiation failed: client connection to 127.12.104.253:39543: connect: Connection refused (error 111)
I20260924 15:08:31.127362 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.253:39543
I20260924 15:08:31.127483 14290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.253:39543 every 8 connection(s)
I20260924 15:08:31.132295 14291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:31.132699 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:08:31.139340 14293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:31.141690 14294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.142140 14291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } has no permanent_uuid. Determining permanent_uuid...
W20260924 15:08:31.145437 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.146076 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:31.146900 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:31.147145 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:31.147343 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262511147324 us; error 0 us; skew 500 ppm
I20260924 15:08:31.147900 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:31.148654 14225 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } attempt: 2
I20260924 15:08:31.153582 12707 webserver.cc:533] Webserver started at http://127.12.104.252:38317/ using document root <none> and password file <none>
I20260924 15:08:31.154397 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:31.154742 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:31.155548 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:31.157474 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-2-root/instance:
uuid: "a57037ec14994dd5ad4f06193c9ec648"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.161837 14291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.165081 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260924 15:08:31.166092 14225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.170918 14303 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260924 15:08:31.172312 14225 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.12.104.252:41327: Network error: Client connection negotiation failed: client connection to 127.12.104.252:41327: connect: Connection refused (error 111)
I20260924 15:08:31.173316 12707 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260924 15:08:31.173661 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-2-root
uuid: "a57037ec14994dd5ad4f06193c9ec648"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.173996 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:31.175302 14291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } has no permanent_uuid. Determining permanent_uuid...
W20260924 15:08:31.183171 14291 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.12.104.252:41327: Network error: Client connection negotiation failed: client connection to 127.12.104.252:41327: connect: Connection refused (error 111)
I20260924 15:08:31.187134 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:31.188442 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.205937 14225 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } attempt: 1
W20260924 15:08:31.213678 14225 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.12.104.252:41327: Network error: Client connection negotiation failed: client connection to 127.12.104.252:41327: connect: Connection refused (error 111)
I20260924 15:08:31.231436 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.252:41327
I20260924 15:08:31.231531 14357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.252:41327 every 8 connection(s)
I20260924 15:08:31.234920 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:08:31.236189 14358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:31.242650 14358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.242766 14291 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } attempt: 1
I20260924 15:08:31.252297 14225 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } attempt: 2
I20260924 15:08:31.260820 14358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.268395 14291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: Bootstrap starting.
I20260924 15:08:31.274361 14358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } has no permanent_uuid. Determining permanent_uuid...
I20260924 15:08:31.275520 14291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:31.278393 14225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: Bootstrap starting.
I20260924 15:08:31.286717 14291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: No bootstrap required, opened a new log
I20260924 15:08:31.286850 14225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:31.290335 14291 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.291278 14291 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:31.291630 14291 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a97f8d8b90b4e6d910032e7df7cff96, State: Initialized, Role: FOLLOWER
I20260924 15:08:31.292552 14291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.293593 14225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: No bootstrap required, opened a new log
I20260924 15:08:31.295193 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.295784 14358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: Bootstrap starting.
I20260924 15:08:31.296203 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.297037 14291 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:31.297122 14225 raft_consensus.cc:359] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.298058 14225 raft_consensus.cc:385] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:31.298451 14225 raft_consensus.cc:740] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Initialized, Role: FOLLOWER
I20260924 15:08:31.299360 14225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.301822 14358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:31.302667 14366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.303719 14366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.305517 14225 sys_catalog.cc:565] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:31.307667 14366 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:31.307991 14358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: No bootstrap required, opened a new log
I20260924 15:08:31.308183 14366 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.310689 14366 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:31.311342 14358 raft_consensus.cc:359] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.312305 14358 raft_consensus.cc:385] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:31.312680 14358 raft_consensus.cc:740] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57037ec14994dd5ad4f06193c9ec648, State: Initialized, Role: FOLLOWER
I20260924 15:08:31.312682 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:31.313537 14266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 0.
I20260924 15:08:31.315218 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96; no voters:
I20260924 15:08:31.316359 14366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:31.316847 14366 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:31.316847 14358 consensus_queue.cc:260] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.317595 14366 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:31.321195 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.326803 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.329597 14375 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:31.332957 14375 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.342219 14358 sys_catalog.cc:565] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:08:31.343933 14200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7" is_pre_election: true
I20260924 15:08:31.345032 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648" is_pre_election: true
I20260924 15:08:31.345853 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 0.
I20260924 15:08:31.347234 14366 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.348434 14200 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a57037ec14994dd5ad4f06193c9ec648 in current term 1: Already voted for candidate 08536d2250d14b76957e5b87dd30a9e7 in this term.
I20260924 15:08:31.350111 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:31.350900 14266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57037ec14994dd5ad4f06193c9ec648 in term 0.
I20260924 15:08:31.352241 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96, a57037ec14994dd5ad4f06193c9ec648; no voters:
I20260924 15:08:31.354781 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:31.355647 14266 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:31.357137 14375 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:31.358551 14375 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:31.360987 14375 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:31.361503 14375 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:31.364615 14266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 1.
I20260924 15:08:31.365724 14366 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:31.367157 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648"
I20260924 15:08:31.368422 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96; no voters:
I20260924 15:08:31.369563 14366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:31.373049 14375 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.374775 14331 raft_consensus.cc:2393] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 08536d2250d14b76957e5b87dd30a9e7 in current term 1: Already voted for candidate a57037ec14994dd5ad4f06193c9ec648 in this term.
I20260924 15:08:31.374771 14366 raft_consensus.cc:697] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 LEADER]: Becoming Leader. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Running, Role: LEADER
I20260924 15:08:31.377056 14375 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 1 election: Requested vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:31.379158 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:31.377902 14366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:31.380600 14266 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a57037ec14994dd5ad4f06193c9ec648 in current term 1: Already voted for candidate 08536d2250d14b76957e5b87dd30a9e7 in this term.
I20260924 15:08:31.381517 14200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7"
I20260924 15:08:31.384268 14307 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96
I20260924 15:08:31.385316 14375 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 15:08:31.392889 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.394177 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:31.405575 14391 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
W20260924 15:08:31.408274 14395 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 15:08:31.410115 14395 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 15:08:31.415599 14391 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:31.422569 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260924 15:08:31.422927 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260924 15:08:31.428238 14331 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 15:08:31.428298 14266 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 15:08:31.431036 14383 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:31.432776 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260924 15:08:31.432824 14383 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 15:08:31.435169 14406 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 15:08:31.435468 14406 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
E20260924 15:08:31.436139 12707 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260924 15:08:31.449622 14408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.453926 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.455736 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.462069 14403 mvcc.cc:204] Tried to move back new op lower bound from 7332915246793293824 to 7332915246622371840. Current Snapshot: MvccSnapshot[applied={T|T < 7332915246793293824}]
I20260924 15:08:31.463056 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.464001 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.464409 14366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.465233 14366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
W20260924 15:08:31.466084 14411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.472702 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.473588 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:31.475453 14391 catalog_manager.cc:1396] Generated new cluster ID: 5c62b95ebc3d4de59a293780a17c5530
I20260924 15:08:31.475965 14391 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:31.477591 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.478744 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:31.481731 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:31.482618 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 15:08:31.488569 14414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.489053 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:31.490283 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:31.490723 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:31.490900 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262511490887 us; error 0 us; skew 500 ppm
I20260924 15:08:31.491479 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:31.494833 12707 webserver.cc:533] Webserver started at http://127.12.104.193:37779/ using document root <none> and password file <none>
I20260924 15:08:31.495908 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:31.496219 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:31.496649 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:31.498509 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "79bbea6671054091bc8e3c57ceda1a68"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.509023 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260924 15:08:31.510169 14391 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:08:31.512277 14391 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:31.520330 14423 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.521435 12707 fs_manager.cc:730] Time spent opening block manager: real 0.008s user 0.004s sys 0.003s
I20260924 15:08:31.521755 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "79bbea6671054091bc8e3c57ceda1a68"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.522017 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:31.537066 14391 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: Generated new TSK 0
I20260924 15:08:31.537861 14391 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:31.546593 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:31.547768 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.549597 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:31.552206 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:31.552452 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.552726 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:31.552917 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.606709 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:38665
I20260924 15:08:31.606838 14488 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:38665 every 8 connection(s)
I20260924 15:08:31.622041 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260924 15:08:31.624836 12707 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260924 15:08:31.637322 14496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:08:31.641026 14497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.646392 14489 heartbeater.cc:344] Connected to a master server at 127.12.104.252:41327
I20260924 15:08:31.646871 14489 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.647972 14489 heartbeater.cc:507] Master 127.12.104.252:41327 requested a full tablet report, sending...
I20260924 15:08:31.649482 14490 heartbeater.cc:344] Connected to a master server at 127.12.104.253:39543
I20260924 15:08:31.650023 14490 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.651465 14490 heartbeater.cc:507] Master 127.12.104.253:39543 requested a full tablet report, sending...
I20260924 15:08:31.651861 14491 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45779
I20260924 15:08:31.652053 14321 ts_manager.cc:194] Registered new tserver with Master: 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665)
W20260924 15:08:31.652119 14499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.653506 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:31.652406 14491 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.655020 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260924 15:08:31.655264 14491 heartbeater.cc:507] Master 127.12.104.254:45779 requested a full tablet report, sending...
W20260924 15:08:31.655397 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:31.655818 14256 ts_manager.cc:194] Registered new tserver with Master: 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665)
I20260924 15:08:31.656024 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262511655992 us; error 0 us; skew 500 ppm
I20260924 15:08:31.657334 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:31.658859 14190 ts_manager.cc:194] Registered new tserver with Master: 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665)
I20260924 15:08:31.661079 14190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51796
I20260924 15:08:31.661433 12707 webserver.cc:533] Webserver started at http://127.12.104.194:37547/ using document root <none> and password file <none>
I20260924 15:08:31.662037 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:31.662346 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:31.662699 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:31.663897 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "77c6c01846a946c49f93ed187dfb4b11"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.669776 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 15:08:31.674451 14504 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.675565 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:08:31.675952 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "77c6c01846a946c49f93ed187dfb4b11"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.676353 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:31.699929 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:31.701385 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.703509 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:31.706667 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:31.706943 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.707222 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:31.707436 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.764308 14491 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:31.793700 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:43571
I20260924 15:08:31.793834 14567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:43571 every 8 connection(s)
I20260924 15:08:31.822809 14568 heartbeater.cc:344] Connected to a master server at 127.12.104.252:41327
I20260924 15:08:31.823390 14568 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.824208 14569 heartbeater.cc:344] Connected to a master server at 127.12.104.253:39543
I20260924 15:08:31.824592 14568 heartbeater.cc:507] Master 127.12.104.252:41327 requested a full tablet report, sending...
I20260924 15:08:31.824702 14569 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.826697 14569 heartbeater.cc:507] Master 127.12.104.253:39543 requested a full tablet report, sending...
I20260924 15:08:31.828341 14321 ts_manager.cc:194] Registered new tserver with Master: 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:31.829160 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260924 15:08:31.831264 14256 ts_manager.cc:194] Registered new tserver with Master: 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
E20260924 15:08:31.833173 12707 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260924 15:08:31.841601 14575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.843683 14570 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45779
I20260924 15:08:31.844281 14570 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:31.846194 14570 heartbeater.cc:507] Master 127.12.104.254:45779 requested a full tablet report, sending...
W20260924 15:08:31.845367 14576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.849990 14190 ts_manager.cc:194] Registered new tserver with Master: 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
W20260924 15:08:31.851094 14578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:08:31.851778 12707 server_base.cc:1061] running on GCE node
I20260924 15:08:31.852016 14190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51800
I20260924 15:08:31.852993 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:08:31.853339 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:08:31.853631 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262511853608 us; error 0 us; skew 500 ppm
I20260924 15:08:31.854363 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:08:31.857396 12707 webserver.cc:533] Webserver started at http://127.12.104.195:38201/ using document root <none> and password file <none>
I20260924 15:08:31.858021 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:08:31.858292 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:08:31.858757 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:08:31.860261 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-2-root/instance:
uuid: "87a4a328946541a9be28ac15366e866d"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.866766 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260924 15:08:31.871001 14583 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.872076 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 15:08:31.872427 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-2-root
uuid: "87a4a328946541a9be28ac15366e866d"
format_stamp: "Formatted at 2026-09-24 15:08:31 on dist-test-slave-9164"
I20260924 15:08:31.872704 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:08:31.890255 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:08:31.892113 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:08:31.894421 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:08:31.897460 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:08:31.897763 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.898039 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:08:31.898243 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:08:31.955610 14570 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:31.991514 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.195:39951
I20260924 15:08:31.991729 14646 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.195:39951 every 8 connection(s)
I20260924 15:08:32.021215 14647 heartbeater.cc:344] Connected to a master server at 127.12.104.252:41327
I20260924 15:08:32.021706 14647 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:32.023882 14647 heartbeater.cc:507] Master 127.12.104.252:41327 requested a full tablet report, sending...
I20260924 15:08:32.026939 14648 heartbeater.cc:344] Connected to a master server at 127.12.104.253:39543
I20260924 15:08:32.027143 14321 ts_manager.cc:194] Registered new tserver with Master: 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.027395 14648 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:32.028316 14648 heartbeater.cc:507] Master 127.12.104.253:39543 requested a full tablet report, sending...
I20260924 15:08:32.031119 14649 heartbeater.cc:344] Connected to a master server at 127.12.104.254:45779
I20260924 15:08:32.031445 14256 ts_manager.cc:194] Registered new tserver with Master: 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.031602 14649 heartbeater.cc:461] Registering TS with master...
I20260924 15:08:32.032902 14649 heartbeater.cc:507] Master 127.12.104.254:45779 requested a full tablet report, sending...
I20260924 15:08:32.035681 14190 ts_manager.cc:194] Registered new tserver with Master: 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.036793 12707 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019091295s
I20260924 15:08:32.037583 14190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51814
I20260924 15:08:32.069832 14189 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260924 15:08:32.072149 14189 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 15:08:32.144444 14649 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:32.197515 14453 tablet_service.cc:1511] Processing CreateTablet for tablet a2fda22f30124c7eaaaff155e72355f3 (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.198166 14452 tablet_service.cc:1511] Processing CreateTablet for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.199386 14453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fda22f30124c7eaaaff155e72355f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.200675 14451 tablet_service.cc:1511] Processing CreateTablet for tablet 47871e968d7a49dabd43afc59d8fe10e (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.201148 14611 tablet_service.cc:1511] Processing CreateTablet for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.202436 14451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47871e968d7a49dabd43afc59d8fe10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.202775 14611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.206336 14532 tablet_service.cc:1511] Processing CreateTablet for tablet a2fda22f30124c7eaaaff155e72355f3 (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.206622 14612 tablet_service.cc:1511] Processing CreateTablet for tablet a2fda22f30124c7eaaaff155e72355f3 (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.208293 14612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fda22f30124c7eaaaff155e72355f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.210896 14452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.210758 14531 tablet_service.cc:1511] Processing CreateTablet for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.208312 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fda22f30124c7eaaaff155e72355f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.215333 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.214983 14530 tablet_service.cc:1511] Processing CreateTablet for tablet 47871e968d7a49dabd43afc59d8fe10e (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.219494 14610 tablet_service.cc:1511] Processing CreateTablet for tablet 47871e968d7a49dabd43afc59d8fe10e (DEFAULT_TABLE table=test-table [id=6e317ddbda864a9cb89fb260bc63c79b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260924 15:08:32.220212 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47871e968d7a49dabd43afc59d8fe10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.221164 14610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47871e968d7a49dabd43afc59d8fe10e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:08:32.283631 14677 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: Bootstrap starting.
I20260924 15:08:32.302963 14679 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: Bootstrap starting.
I20260924 15:08:32.303750 14678 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: Bootstrap starting.
I20260924 15:08:32.310067 14677 tablet_bootstrap.cc:654] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.310287 14678 tablet_bootstrap.cc:654] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.311467 14679 tablet_bootstrap.cc:654] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.319641 14678 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: No bootstrap required, opened a new log
I20260924 15:08:32.320245 14678 ts_tablet_manager.cc:1403] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.001s
I20260924 15:08:32.320966 14677 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: No bootstrap required, opened a new log
I20260924 15:08:32.322593 14677 ts_tablet_manager.cc:1403] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.009s
I20260924 15:08:32.323892 14679 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: No bootstrap required, opened a new log
I20260924 15:08:32.324455 14679 ts_tablet_manager.cc:1403] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.006s
I20260924 15:08:32.325201 14678 raft_consensus.cc:359] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.326010 14678 raft_consensus.cc:385] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.325980 14677 raft_consensus.cc:359] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.326644 14678 raft_consensus.cc:740] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a4a328946541a9be28ac15366e866d, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.327008 14677 raft_consensus.cc:385] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.327428 14677 raft_consensus.cc:740] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c6c01846a946c49f93ed187dfb4b11, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.327672 14678 consensus_queue.cc:260] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.328256 14677 consensus_queue.cc:260] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.328442 14679 raft_consensus.cc:359] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.330093 14679 raft_consensus.cc:385] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.330585 14679 raft_consensus.cc:740] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.331516 14677 ts_tablet_manager.cc:1434] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260924 15:08:32.331909 14679 consensus_queue.cc:260] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.342077 14677 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: Bootstrap starting.
I20260924 15:08:32.362893 14677 tablet_bootstrap.cc:654] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.366063 14678 ts_tablet_manager.cc:1434] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: Time spent starting tablet: real 0.044s user 0.011s sys 0.011s
I20260924 15:08:32.367311 14678 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: Bootstrap starting.
I20260924 15:08:32.369030 14679 ts_tablet_manager.cc:1434] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: Time spent starting tablet: real 0.044s user 0.017s sys 0.016s
I20260924 15:08:32.380086 14679 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: Bootstrap starting.
I20260924 15:08:32.391532 14679 tablet_bootstrap.cc:654] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.395885 14684 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.396845 14684 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.401288 14685 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.407187 14677 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: No bootstrap required, opened a new log
I20260924 15:08:32.408140 14677 ts_tablet_manager.cc:1403] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: Time spent bootstrapping tablet: real 0.066s user 0.011s sys 0.019s
I20260924 15:08:32.408131 14678 tablet_bootstrap.cc:654] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.415056 14684 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.403995 14685 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.419762 14395 catalog_manager.cc:1308] Loaded cluster ID: 5c62b95ebc3d4de59a293780a17c5530
I20260924 15:08:32.419483 14677 raft_consensus.cc:359] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.422750 14677 raft_consensus.cc:385] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.424599 14677 raft_consensus.cc:740] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c6c01846a946c49f93ed187dfb4b11, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.424448 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.425889 14683 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.422565 14395 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: loading cluster ID for follower catalog manager: success
I20260924 15:08:32.426510 14683 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
W20260924 15:08:32.428011 14586 leader_election.cc:343] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Illegal state: must be running to vote when last-logged opid is not known
I20260924 15:08:32.428400 14677 consensus_queue.cc:260] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.431888 14683 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.462767 14683 raft_consensus.cc:493] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.463637 14683 raft_consensus.cc:515] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.466387 14685 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.491379 14395 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: acquiring CA information for follower catalog manager: success
I20260924 15:08:32.495759 14677 ts_tablet_manager.cc:1434] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: Time spent starting tablet: real 0.086s user 0.012s sys 0.017s
I20260924 15:08:32.499539 14679 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: No bootstrap required, opened a new log
I20260924 15:08:32.500452 14679 ts_tablet_manager.cc:1403] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: Time spent bootstrapping tablet: real 0.121s user 0.007s sys 0.017s
I20260924 15:08:32.506657 14543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:32.548985 14543 raft_consensus.cc:2468] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a4a328946541a9be28ac15366e866d in term 0.
I20260924 15:08:32.549788 14395 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 15:08:32.550074 14683 leader_election.cc:290] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.564254 14677 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: Bootstrap starting.
I20260924 15:08:32.575613 14585 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d; no voters: 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:32.581897 14684 raft_consensus.cc:2804] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:32.589702 14684 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:32.590273 14684 raft_consensus.cc:3060] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.593245 14679 raft_consensus.cc:359] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.594044 14679 raft_consensus.cc:385] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.594280 14679 raft_consensus.cc:740] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.600391 14406 catalog_manager.cc:1308] Loaded cluster ID: 5c62b95ebc3d4de59a293780a17c5530
I20260924 15:08:32.607900 14678 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: No bootstrap required, opened a new log
I20260924 15:08:32.604394 14406 catalog_manager.cc:1601] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: loading cluster ID for follower catalog manager: success
I20260924 15:08:32.612268 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.613190 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.608618 14678 ts_tablet_manager.cc:1403] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: Time spent bootstrapping tablet: real 0.242s user 0.043s sys 0.034s
I20260924 15:08:32.624029 14622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.625449 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.625973 14678 raft_consensus.cc:359] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.626866 14678 raft_consensus.cc:385] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.627872 14678 raft_consensus.cc:740] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a4a328946541a9be28ac15366e866d, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.628818 14678 consensus_queue.cc:260] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
W20260924 15:08:32.632364 14508 leader_election.cc:343] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951): Illegal state: must be running to vote when last-logged opid is not known
W20260924 15:08:32.635612 14507 leader_election.cc:343] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Illegal state: must be running to vote when last-logged opid is not known
I20260924 15:08:32.640110 14678 ts_tablet_manager.cc:1434] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: Time spent starting tablet: real 0.017s user 0.009s sys 0.004s
I20260924 15:08:32.640057 14543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:32.641579 14678 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: Bootstrap starting.
I20260924 15:08:32.648164 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.648957 14621 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79bbea6671054091bc8e3c57ceda1a68 in term 0.
I20260924 15:08:32.649531 14678 tablet_bootstrap.cc:654] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.613431 14679 consensus_queue.cc:260] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.651517 14462 raft_consensus.cc:2468] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77c6c01846a946c49f93ed187dfb4b11 in term 0.
W20260924 15:08:32.652065 14425 leader_election.cc:343] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571): Illegal state: must be running to vote when last-logged opid is not known
I20260924 15:08:32.656418 14679 ts_tablet_manager.cc:1434] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: Time spent starting tablet: real 0.155s user 0.007s sys 0.005s
I20260924 15:08:32.659544 14684 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.670053 14678 tablet_bootstrap.cc:492] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: No bootstrap required, opened a new log
I20260924 15:08:32.670775 14678 ts_tablet_manager.cc:1403] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: Time spent bootstrapping tablet: real 0.029s user 0.016s sys 0.012s
I20260924 15:08:32.671469 14622 raft_consensus.cc:2393] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 in current term 1: Already voted for candidate 87a4a328946541a9be28ac15366e866d in this term.
I20260924 15:08:32.672005 14406 catalog_manager.cc:1623] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: acquiring CA information for follower catalog manager: success
I20260924 15:08:32.675199 14543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11"
I20260924 15:08:32.665567 14677 tablet_bootstrap.cc:654] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.675174 14678 raft_consensus.cc:359] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.676450 14678 raft_consensus.cc:385] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.661574 14679 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: Bootstrap starting.
I20260924 15:08:32.677093 14678 raft_consensus.cc:740] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a4a328946541a9be28ac15366e866d, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.676131 14543 raft_consensus.cc:3060] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.678262 14678 consensus_queue.cc:260] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.680799 14678 ts_tablet_manager.cc:1434] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260924 15:08:32.662757 14507 leader_election.cc:304] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68; no voters: 87a4a328946541a9be28ac15366e866d
I20260924 15:08:32.666409 14427 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11
I20260924 15:08:32.684540 14685 raft_consensus.cc:2804] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:32.684706 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.685063 14685 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:32.685472 14685 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.685626 14621 raft_consensus.cc:2393] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 in current term 1: Already voted for candidate 87a4a328946541a9be28ac15366e866d in this term.
I20260924 15:08:32.674261 14684 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 election: Requested vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.685932 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68"
I20260924 15:08:32.687810 14683 raft_consensus.cc:2804] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:32.688570 14508 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11; no voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d
I20260924 15:08:32.688792 14683 raft_consensus.cc:493] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:32.689455 14683 raft_consensus.cc:3060] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.690634 14696 raft_consensus.cc:493] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 15:08:32.693936 14586 leader_election.cc:343] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Illegal state: must be running to vote when last-logged opid is not known
I20260924 15:08:32.694475 14696 raft_consensus.cc:515] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.703033 14696 leader_election.cc:290] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.704890 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.705123 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:32.705720 14462 raft_consensus.cc:2468] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a4a328946541a9be28ac15366e866d in term 0.
I20260924 15:08:32.712618 14586 leader_election.cc:304] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d; no voters:
I20260924 15:08:32.713057 14685 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.713660 14696 raft_consensus.cc:2804] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:08:32.714088 14696 raft_consensus.cc:493] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:32.714545 14696 raft_consensus.cc:3060] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.716143 14543 raft_consensus.cc:2468] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a4a328946541a9be28ac15366e866d in term 1.
I20260924 15:08:32.716320 14685 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 election: Requested vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.717725 14689 raft_consensus.cc:2749] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 15:08:32.719842 14585 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d; no voters: 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:32.720245 14406 catalog_manager.cc:1651] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 15:08:32.720918 14684 raft_consensus.cc:2804] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:32.724735 14679 tablet_bootstrap.cc:654] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: Neither blocks nor log segments found. Creating new log.
I20260924 15:08:32.731078 14543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11"
I20260924 15:08:32.731716 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d"
I20260924 15:08:32.734786 14621 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.737032 14679 tablet_bootstrap.cc:492] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: No bootstrap required, opened a new log
I20260924 15:08:32.737627 14679 ts_tablet_manager.cc:1403] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.013s
I20260924 15:08:32.737349 14696 raft_consensus.cc:515] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.733078 14684 raft_consensus.cc:697] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 LEADER]: Becoming Leader. State: Replica: 87a4a328946541a9be28ac15366e866d, State: Running, Role: LEADER
I20260924 15:08:32.740077 14684 consensus_queue.cc:237] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.742935 14696 leader_election.cc:290] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 election: Requested vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:32.743435 14540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11"
I20260924 15:08:32.746991 14621 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79bbea6671054091bc8e3c57ceda1a68 in term 1.
W20260924 15:08:32.745666 14425 leader_election.cc:343] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571): Illegal state: must be running to vote when last-logged opid is not known
I20260924 15:08:32.748092 14679 raft_consensus.cc:359] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.737311 14683 raft_consensus.cc:515] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.750289 14679 raft_consensus.cc:385] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.751088 14542 raft_consensus.cc:2393] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87a4a328946541a9be28ac15366e866d in current term 1: Already voted for candidate 77c6c01846a946c49f93ed187dfb4b11 in this term.
I20260924 15:08:32.750872 14679 raft_consensus.cc:740] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.755172 14679 consensus_queue.cc:260] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.758472 14683 leader_election.cc:290] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 election: Requested vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.759698 14427 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11
I20260924 15:08:32.764837 14685 raft_consensus.cc:2804] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:32.844987 14677 tablet_bootstrap.cc:492] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: No bootstrap required, opened a new log
I20260924 15:08:32.845746 14677 ts_tablet_manager.cc:1403] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: Time spent bootstrapping tablet: real 0.294s user 0.013s sys 0.005s
I20260924 15:08:32.841985 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68"
I20260924 15:08:32.853181 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d"
I20260924 15:08:32.855118 14621 raft_consensus.cc:2393] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 in current term 1: Already voted for candidate 87a4a328946541a9be28ac15366e866d in this term.
I20260924 15:08:32.851596 14677 raft_consensus.cc:359] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.857563 14463 raft_consensus.cc:3060] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Advancing to term 1
W20260924 15:08:32.862262 14586 consensus_peers.cc:597] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d -> Peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Couldn't send request to peer 79bbea6671054091bc8e3c57ceda1a68. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:08:32.861518 14677 raft_consensus.cc:385] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:08:32.863365 14677 raft_consensus.cc:740] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c6c01846a946c49f93ed187dfb4b11, State: Initialized, Role: FOLLOWER
I20260924 15:08:32.864460 14677 consensus_queue.cc:260] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.833294 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68"
I20260924 15:08:32.871483 14586 leader_election.cc:304] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:32.871860 14679 ts_tablet_manager.cc:1434] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: Time spent starting tablet: real 0.134s user 0.038s sys 0.066s
I20260924 15:08:32.869719 14677 ts_tablet_manager.cc:1434] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: Time spent starting tablet: real 0.024s user 0.007s sys 0.003s
I20260924 15:08:32.873207 14696 raft_consensus.cc:2749] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 15:08:32.878364 14685 raft_consensus.cc:697] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 LEADER]: Becoming Leader. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Running, Role: LEADER
I20260924 15:08:32.878927 14702 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.879522 14702 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.879581 14685 consensus_queue.cc:237] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.884130 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:32.884859 14542 raft_consensus.cc:2393] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79bbea6671054091bc8e3c57ceda1a68 in current term 1: Already voted for candidate 87a4a328946541a9be28ac15366e866d in this term.
I20260924 15:08:32.884686 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.889107 14463 raft_consensus.cc:2468] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77c6c01846a946c49f93ed187dfb4b11 in term 1.
I20260924 15:08:32.890084 14702 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:32.892978 14507 leader_election.cc:304] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68; no voters: 87a4a328946541a9be28ac15366e866d
I20260924 15:08:32.894110 14683 raft_consensus.cc:2804] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:08:32.894860 14683 raft_consensus.cc:697] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 LEADER]: Becoming Leader. State: Replica: 77c6c01846a946c49f93ed187dfb4b11, State: Running, Role: LEADER
I20260924 15:08:32.895439 14684 consensus_queue.cc:1048] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:32.895946 14427 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68; no voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d
I20260924 15:08:32.895790 14683 consensus_queue.cc:237] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:32.901636 14702 raft_consensus.cc:3060] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.897025 14187 catalog_manager.cc:5838] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a4a328946541a9be28ac15366e866d (127.12.104.195). New cstate: current_term: 1 leader_uuid: "87a4a328946541a9be28ac15366e866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } health_report { overall_health: HEALTHY } } }
I20260924 15:08:32.918202 14702 raft_consensus.cc:2749] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 15:08:32.929775 14190 catalog_manager.cc:5838] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193). New cstate: current_term: 1 leader_uuid: "79bbea6671054091bc8e3c57ceda1a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } health_report { overall_health: UNKNOWN } } }
I20260924 15:08:32.956224 14683 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.956811 14683 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.960033 14683 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665)
I20260924 15:08:32.961618 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.964169 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:32.965031 14621 raft_consensus.cc:2393] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 in current term 1: Already voted for candidate 79bbea6671054091bc8e3c57ceda1a68 in this term.
I20260924 15:08:32.966622 14508 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11; no voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d
I20260924 15:08:32.970561 14683 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:08:32.981083 14689 consensus_queue.cc:1048] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 15:08:32.986208 14683 raft_consensus.cc:2749] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 15:08:32.993992 14696 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:32.994626 14696 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:32.998492 14461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:32.999003 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:32.999729 14542 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a4a328946541a9be28ac15366e866d in term 1.
I20260924 15:08:33.000326 14696 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:33.001127 14585 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d; no voters:
I20260924 15:08:33.002169 14696 raft_consensus.cc:2804] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:08:33.002633 14696 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:33.003075 14696 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:33.011387 14702 consensus_queue.cc:1048] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 15:08:33.013257 14685 consensus_queue.cc:1048] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 15:08:33.065666 14689 consensus_queue.cc:1048] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:08:33.071547 14696 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.119234 14621 raft_consensus.cc:1240] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Rejecting Update request from peer 79bbea6671054091bc8e3c57ceda1a68 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260924 15:08:33.121667 14462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68"
I20260924 15:08:33.121860 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11"
I20260924 15:08:33.123167 14696 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 election: Requested vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:33.198616 14702 consensus_queue.cc:1059] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260924 15:08:33.204303 14705 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87a4a328946541a9be28ac15366e866d)
I20260924 15:08:33.206519 14728 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:33.224789 14703 raft_consensus.cc:3055] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 15:08:33.224973 14705 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:33.226382 14586 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:33.229857 14703 raft_consensus.cc:740] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Running, Role: LEADER
I20260924 15:08:33.233297 14699 raft_consensus.cc:2749] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 15:08:33.236943 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:33.234140 14705 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:33.236743 14622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:33.240962 14728 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.242815 14427 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68; no voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d
I20260924 15:08:33.248303 14703 consensus_queue.cc:260] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.250351 14705 raft_consensus.cc:2749] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 15:08:33.246940 14729 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:33.254211 14729 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.259452 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648" is_pre_election: true
I20260924 15:08:33.260370 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a97f8d8b90b4e6d910032e7df7cff96 in term 1.
I20260924 15:08:33.262135 14265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:33.263131 14265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57037ec14994dd5ad4f06193c9ec648 in term 1.
I20260924 15:08:33.263944 14200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7" is_pre_election: true
I20260924 15:08:33.264804 14241 leader_election.cc:304] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96, a57037ec14994dd5ad4f06193c9ec648; no voters:
I20260924 15:08:33.262123 14728 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:33.264401 14729 leader_election.cc:290] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:33.268527 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7" is_pre_election: true
I20260924 15:08:33.269677 14729 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:08:33.255558 14703 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:33.270619 14186 catalog_manager.cc:5838] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194). New cstate: current_term: 1 leader_uuid: "77c6c01846a946c49f93ed187dfb4b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } health_report { overall_health: UNKNOWN } } }
I20260924 15:08:33.273065 14696 raft_consensus.cc:493] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 77c6c01846a946c49f93ed187dfb4b11)
I20260924 15:08:33.273736 14696 raft_consensus.cc:515] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:33.277642 14729 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:33.278255 14729 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:33.278087 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96, a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7
I20260924 15:08:33.280158 14696 leader_election.cc:290] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:33.281157 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:33.281260 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47871e968d7a49dabd43afc59d8fe10e" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:33.283708 14586 leader_election.cc:304] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:33.284004 14728 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:08:33.284631 14728 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:33.284842 14696 raft_consensus.cc:2749] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 15:08:33.285127 14728 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:08:33.294907 14728 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.297104 14729 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.299675 14331 raft_consensus.cc:1240] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Rejecting Update request from peer 08536d2250d14b76957e5b87dd30a9e7 for earlier term 1. Current term is 2. Ops: [1.9-1.9]
I20260924 15:08:33.303489 14728 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 2 election: Requested vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:33.304689 14729 leader_election.cc:290] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 2 election: Requested vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:33.305425 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7"
I20260924 15:08:33.306146 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648"
I20260924 15:08:33.307505 14331 raft_consensus.cc:2393] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a97f8d8b90b4e6d910032e7df7cff96 in current term 2: Already voted for candidate a57037ec14994dd5ad4f06193c9ec648 in this term.
I20260924 15:08:33.307593 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7"
I20260924 15:08:33.309087 14241 leader_election.cc:304] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96; no voters: 08536d2250d14b76957e5b87dd30a9e7, a57037ec14994dd5ad4f06193c9ec648
I20260924 15:08:33.312497 14265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:33.309266 14663 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 }, Status: INVALID_TERM, Last received: 1.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 15:08:33.314507 14366 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 15:08:33.314945 14366 raft_consensus.cc:740] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Running, Role: LEADER
I20260924 15:08:33.314752 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96
I20260924 15:08:33.315436 14265 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 2 FOLLOWER]: Rejecting Update request from peer 08536d2250d14b76957e5b87dd30a9e7 for earlier term 1. Current term is 2. Ops: [1.9-1.9]
I20260924 15:08:33.316133 14366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.317279 14729 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 15:08:33.317596 14728 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 15:08:33.317404 14366 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 1 FOLLOWER]: Advancing to term 2
W20260924 15:08:33.317407 14663 consensus_queue.cc:1175] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" responder_term: 2 status { last_received { term: 1 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 08536d2250d14b76957e5b87dd30a9e7 for earlier term 1. Current term is 2. Ops: [1.9-1.9]" } } last_received_current_leader { term: 0 index: 0 } }
W20260924 15:08:33.359985 14658 master_proxy_rpc.cc:203] Re-attempting IsCreateTableDone request to leader Master (127.12.104.254:45779)
I20260924 15:08:33.442034 14703 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:33.442727 14703 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.446509 14703 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:33.446688 14622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:33.447696 14622 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79bbea6671054091bc8e3c57ceda1a68 in term 2.
I20260924 15:08:33.447916 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:33.450325 14427 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11
I20260924 15:08:33.451318 14703 raft_consensus.cc:2804] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 15:08:33.451696 14703 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:33.452122 14703 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.456104 14731 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79bbea6671054091bc8e3c57ceda1a68)
I20260924 15:08:33.456832 14731 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.459082 14703 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.460632 14622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:33.460942 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "77c6c01846a946c49f93ed187dfb4b11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:33.461035 14731 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665)
I20260924 15:08:33.461755 14622 raft_consensus.cc:2393] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 in current term 2: Already voted for candidate 87a4a328946541a9be28ac15366e866d in this term.
I20260924 15:08:33.461828 14703 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 3 election: Requested vote from peers 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:33.461974 14463 raft_consensus.cc:2368] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 for earlier term 2. Current term is 3.
I20260924 15:08:33.463986 14622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d"
I20260924 15:08:33.465106 14542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11"
I20260924 15:08:33.464586 14507 leader_election.cc:400] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665) with higher term. Message: Invalid argument: T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 for earlier term 2. Current term is 3.
I20260924 15:08:33.464635 14622 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.465981 14507 leader_election.cc:403] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260924 15:08:33.465981 14542 raft_consensus.cc:3060] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.473991 14622 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79bbea6671054091bc8e3c57ceda1a68 in term 3.
I20260924 15:08:33.475421 14427 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68, 87a4a328946541a9be28ac15366e866d; no voters:
I20260924 15:08:33.476313 14703 raft_consensus.cc:2804] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 15:08:33.476682 14542 raft_consensus.cc:2468] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79bbea6671054091bc8e3c57ceda1a68 in term 3.
I20260924 15:08:33.477010 14703 raft_consensus.cc:697] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 LEADER]: Becoming Leader. State: Replica: 79bbea6671054091bc8e3c57ceda1a68, State: Running, Role: LEADER
I20260924 15:08:33.477466 14683 raft_consensus.cc:2749] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665) with higher term. Message: Invalid argument: T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 77c6c01846a946c49f93ed187dfb4b11 for earlier term 2. Current term is 3.
I20260924 15:08:33.478269 14703 consensus_queue.cc:237] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:33.552070 14366 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:33.552584 14366 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.554270 14366 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:33.555099 14265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:33.555420 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648" is_pre_election: true
I20260924 15:08:33.555858 14265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 2.
I20260924 15:08:33.556010 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 2.
I20260924 15:08:33.557189 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96; no voters:
I20260924 15:08:33.558074 14366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 15:08:33.558691 14366 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:33.558998 14366 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.566372 14366 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.569315 14366 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:33.570233 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648"
I20260924 15:08:33.570713 14265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:33.571084 14331 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.571408 14265 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 2 FOLLOWER]: Advancing to term 3
I20260924 15:08:33.580093 14265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 3.
I20260924 15:08:33.580093 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 3.
I20260924 15:08:33.582391 14174 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, a57037ec14994dd5ad4f06193c9ec648; no voters:
I20260924 15:08:33.583376 14366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 15:08:33.584034 14366 raft_consensus.cc:697] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 LEADER]: Becoming Leader. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Running, Role: LEADER
I20260924 15:08:33.584800 14366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:33.591135 14663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.592357 14663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:33.594408 14622 raft_consensus.cc:1275] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Refusing update from remote peer 79bbea6671054091bc8e3c57ceda1a68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 15:08:33.597929 14703 consensus_queue.cc:1048] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 15:08:33.603152 14542 raft_consensus.cc:1275] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 3 FOLLOWER]: Refusing update from remote peer 79bbea6671054091bc8e3c57ceda1a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 15:08:33.604883 14703 consensus_queue.cc:1048] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 15:08:33.683349 14331 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20260924 15:08:33.685401 14663 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 15:08:33.692833 14728 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.693842 14728 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:33.694475 14265 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20260924 15:08:33.696401 14663 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 15:08:33.700454 14663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.702368 14663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:33.706408 14729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.707513 14729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:33.711668 14366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.712518 14366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:33.715391 14728 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.716445 14728 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:33.724745 14732 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:08:33.732009 14729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:33.732903 14729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:33.735949 14732 catalog_manager.cc:692] Loaded metadata for table test-table [id=6e317ddbda864a9cb89fb260bc63c79b]
I20260924 15:08:33.741019 14732 tablet_loader.cc:96] loaded metadata for tablet 47871e968d7a49dabd43afc59d8fe10e (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:08:33.743209 14732 tablet_loader.cc:96] loaded metadata for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:08:33.744745 14732 tablet_loader.cc:96] loaded metadata for tablet a2fda22f30124c7eaaaff155e72355f3 (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:08:33.746491 14732 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:08:33.752065 14732 catalog_manager.cc:1308] Loaded cluster ID: 5c62b95ebc3d4de59a293780a17c5530
I20260924 15:08:33.752434 14732 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:08:33.756561 14732 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:08:33.759625 14732 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: Loaded TSK: 0
I20260924 15:08:33.760914 14732 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:08:33.818683 14570 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:33.824131 14491 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:33.836161 14185 catalog_manager.cc:5838] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "79bbea6671054091bc8e3c57ceda1a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } }
I20260924 15:08:33.841224 14649 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
I20260924 15:08:41.522850 14776 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:41.523510 14776 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.526834 14776 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:41.581092 14766 raft_consensus.cc:493] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87a4a328946541a9be28ac15366e866d)
I20260924 15:08:41.591218 14777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:41.591926 14777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.594779 14777 leader_election.cc:290] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:41.617189 14775 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 79bbea6671054091bc8e3c57ceda1a68)
I20260924 15:08:41.617352 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7" is_pre_election: true
I20260924 15:08:41.618299 14775 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:08:41.620844 14775 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:08:41.637218 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7" is_pre_election: true
I20260924 15:08:41.581737 14766 raft_consensus.cc:515] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } } peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } }
I20260924 15:08:41.679397 14766 leader_election.cc:290] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571), 87a4a328946541a9be28ac15366e866d (127.12.104.195:39951)
I20260924 15:08:41.681959 14540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:41.695876 14540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "77c6c01846a946c49f93ed187dfb4b11" is_pre_election: true
I20260924 15:08:41.699672 14265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:41.699754 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648" is_pre_election: true
I20260924 15:08:41.700580 14265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57037ec14994dd5ad4f06193c9ec648 in term 3.
I20260924 15:08:41.700834 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a97f8d8b90b4e6d910032e7df7cff96 in term 3.
I20260924 15:08:41.703604 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96, a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7
I20260924 15:08:41.703949 14241 leader_election.cc:304] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96, a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7
I20260924 15:08:41.705161 14777 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 15:08:41.705591 14777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:41.705998 14777 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 3 FOLLOWER]: Advancing to term 4
I20260924 15:08:41.710100 14463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2fda22f30124c7eaaaff155e72355f3" candidate_uuid: "87a4a328946541a9be28ac15366e866d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "79bbea6671054091bc8e3c57ceda1a68" is_pre_election: true
I20260924 15:08:41.715462 14776 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 15:08:41.715950 14776 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:08:41.716382 14776 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 3 FOLLOWER]: Advancing to term 4
I20260924 15:08:41.718003 14621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e648c07d8ee4c5bb1062d8d3b6a9bea" candidate_uuid: "79bbea6671054091bc8e3c57ceda1a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "87a4a328946541a9be28ac15366e866d" is_pre_election: true
I20260924 15:08:41.718492 14777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.719903 14266 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Rejecting Update request from peer 08536d2250d14b76957e5b87dd30a9e7 for earlier term 3. Current term is 4. Ops: []
I20260924 15:08:41.720144 14586 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:08:41.722589 14771 raft_consensus.cc:2749] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 15:08:41.723301 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7"
I20260924 15:08:41.723295 14427 leader_election.cc:304] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79bbea6671054091bc8e3c57ceda1a68; no voters: 77c6c01846a946c49f93ed187dfb4b11, 87a4a328946541a9be28ac15366e866d
I20260924 15:08:41.724766 14766 raft_consensus.cc:2749] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 15:08:41.725445 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648"
I20260924 15:08:41.726524 14777 leader_election.cc:290] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 4 election: Requested vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:41.726754 14241 leader_election.cc:304] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a97f8d8b90b4e6d910032e7df7cff96; no voters: 08536d2250d14b76957e5b87dd30a9e7, a57037ec14994dd5ad4f06193c9ec648
I20260924 15:08:41.727559 14777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 15:08:41.727741 14776 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.729821 14776 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 4 election: Requested vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
I20260924 15:08:41.730463 14779 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 }, Status: INVALID_TERM, Last received: 3.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 15:08:41.730909 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:41.730729 14199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a57037ec14994dd5ad4f06193c9ec648" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "08536d2250d14b76957e5b87dd30a9e7"
I20260924 15:08:41.731747 14266 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a57037ec14994dd5ad4f06193c9ec648 in current term 4: Already voted for candidate 0a97f8d8b90b4e6d910032e7df7cff96 in this term.
I20260924 15:08:41.732075 14774 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 15:08:41.732443 14774 raft_consensus.cc:740] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Running, Role: LEADER
I20260924 15:08:41.733006 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96
I20260924 15:08:41.733307 14774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.733857 14776 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 15:08:41.734632 14774 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 3 FOLLOWER]: Advancing to term 4
I20260924 15:08:41.915570 14774 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:08:41.916199 14774 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.919432 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" is_pre_election: true
I20260924 15:08:41.920259 14266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 4.
I20260924 15:08:41.920564 14774 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:41.921663 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96; no voters:
I20260924 15:08:41.921820 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648" is_pre_election: true
I20260924 15:08:41.922554 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 4.
I20260924 15:08:41.923071 14774 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 15:08:41.923501 14774 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:08:41.924216 14774 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 4 FOLLOWER]: Advancing to term 5
I20260924 15:08:41.930692 14774 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.933308 14774 leader_election.cc:290] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543), a57037ec14994dd5ad4f06193c9ec648 (127.12.104.252:41327)
I20260924 15:08:41.933584 14266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "0a97f8d8b90b4e6d910032e7df7cff96"
I20260924 15:08:41.934307 14266 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 4 FOLLOWER]: Advancing to term 5
I20260924 15:08:41.934520 14331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08536d2250d14b76957e5b87dd30a9e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "a57037ec14994dd5ad4f06193c9ec648"
I20260924 15:08:41.935220 14331 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 4 FOLLOWER]: Advancing to term 5
I20260924 15:08:41.941372 14331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 5.
I20260924 15:08:41.942165 14266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08536d2250d14b76957e5b87dd30a9e7 in term 5.
I20260924 15:08:41.942687 14174 leader_election.cc:304] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08536d2250d14b76957e5b87dd30a9e7, a57037ec14994dd5ad4f06193c9ec648; no voters:
I20260924 15:08:41.943661 14774 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 15:08:41.944284 14774 raft_consensus.cc:697] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 5 LEADER]: Becoming Leader. State: Replica: 08536d2250d14b76957e5b87dd30a9e7, State: Running, Role: LEADER
I20260924 15:08:41.945197 14774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:08:41.951081 14779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:41.951727 14779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:42.034092 14331 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 5 index: 12. (index mismatch)
I20260924 15:08:42.035748 14779 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 15:08:42.041810 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.042784 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:42.046825 14779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.047827 14779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
I20260924 15:08:42.048069 14266 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 5 FOLLOWER]: Refusing update from remote peer 08536d2250d14b76957e5b87dd30a9e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 5 index: 12. (index mismatch)
I20260924 15:08:42.049222 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.050038 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:42.049993 14774 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 15:08:42.057852 14787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.058977 14787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:42.059269 14777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08536d2250d14b76957e5b87dd30a9e7. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.060067 14777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 15:08:42.146276 14774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "08536d2250d14b76957e5b87dd30a9e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } } }
I20260924 15:08:42.147315 14774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [sys.catalog]: This master's current role is: LEADER
W20260924 15:08:42.582551 14643 debug-util.cc:398] Leaking SignalData structure 0x7b0800093340 after lost signal to thread 12710
W20260924 15:08:42.583873 14643 debug-util.cc:398] Leaking SignalData structure 0x7b0800033780 after lost signal to thread 14224
W20260924 15:08:42.584771 14643 debug-util.cc:398] Leaking SignalData structure 0x7b0800060360 after lost signal to thread 14290
W20260924 15:08:42.585819 14643 debug-util.cc:398] Leaking SignalData structure 0x7b0800215ac0 after lost signal to thread 14357
W20260924 15:08:42.587446 14643 debug-util.cc:398] Leaking SignalData structure 0x7b080014cd60 after lost signal to thread 14488
W20260924 15:08:42.588587 14643 debug-util.cc:398] Leaking SignalData structure 0x7b080021ad40 after lost signal to thread 14567
W20260924 15:08:42.589762 14643 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6040 after lost signal to thread 14646
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:784: Failure
Expected equality of these values:
live_row_count
Which is: 22
table_info->GetMetrics()->live_row_count->value()
Which is: 0
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:768: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260924 15:09:04.422369 14781 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:786: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 784, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
Actual: it does.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:823: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260924 15:09:04.436115 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:04.444079 14781 catalog_manager.cc:692] Loaded metadata for table test-table [id=6e317ddbda864a9cb89fb260bc63c79b]
I20260924 15:09:04.454982 14781 tablet_loader.cc:96] loaded metadata for tablet 47871e968d7a49dabd43afc59d8fe10e (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:09:04.456269 14781 tablet_loader.cc:96] loaded metadata for tablet 4e648c07d8ee4c5bb1062d8d3b6a9bea (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:09:04.457667 14781 tablet_loader.cc:96] loaded metadata for tablet a2fda22f30124c7eaaaff155e72355f3 (table test-table [id=6e317ddbda864a9cb89fb260bc63c79b])
I20260924 15:09:04.459856 14781 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:04.466372 14781 catalog_manager.cc:1308] Loaded cluster ID: 5c62b95ebc3d4de59a293780a17c5530
I20260924 15:09:04.466777 14781 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
W20260924 15:09:04.468429 14586 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
I20260924 15:09:04.487664 14781 catalog_manager.cc:1553] Loading token signing keys...
W20260924 15:09:04.488380 14586 consensus_peers.cc:597] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d -> Peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Couldn't send request to peer 79bbea6671054091bc8e3c57ceda1a68. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 15:09:04.498148 14507 consensus_peers.cc:597] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 -> Peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Couldn't send request to peer 79bbea6671054091bc8e3c57ceda1a68. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:04.499748 14781 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: Loaded TSK: 0
I20260924 15:09:04.501050 14781 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:04.513118 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:04.513907 12707 tablet_replica.cc:333] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68: stopping tablet replica
I20260924 15:09:04.514673 12707 raft_consensus.cc:2243] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.515369 12707 raft_consensus.cc:2272] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.517756 12707 tablet_replica.cc:333] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68: stopping tablet replica
I20260924 15:09:04.518265 12707 raft_consensus.cc:2243] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.518798 12707 raft_consensus.cc:2272] T 47871e968d7a49dabd43afc59d8fe10e P 79bbea6671054091bc8e3c57ceda1a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.520684 12707 tablet_replica.cc:333] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68: stopping tablet replica
I20260924 15:09:04.521155 12707 raft_consensus.cc:2243] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 LEADER]: Raft consensus shutting down.
I20260924 15:09:04.522552 12707 raft_consensus.cc:2272] T a2fda22f30124c7eaaaff155e72355f3 P 79bbea6671054091bc8e3c57ceda1a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.547689 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:04.565814 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:09:04.593319 14649 heartbeater.cc:499] Master 127.12.104.254:45779 was elected leader, sending a full tablet report...
W20260924 15:09:04.617066 14585 consensus_peers.cc:597] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d -> Peer 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571): Couldn't send request to peer 77c6c01846a946c49f93ed187dfb4b11. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:04.631458 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:04.632463 12707 tablet_replica.cc:333] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11: stopping tablet replica
I20260924 15:09:04.633190 12707 raft_consensus.cc:2243] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.633747 12707 raft_consensus.cc:2272] T a2fda22f30124c7eaaaff155e72355f3 P 77c6c01846a946c49f93ed187dfb4b11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.636984 12707 tablet_replica.cc:333] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11: stopping tablet replica
I20260924 15:09:04.637748 12707 raft_consensus.cc:2243] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:04.638796 12707 raft_consensus.cc:2272] T 47871e968d7a49dabd43afc59d8fe10e P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.640936 12707 tablet_replica.cc:333] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11: stopping tablet replica
I20260924 15:09:04.641470 12707 raft_consensus.cc:2243] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.641943 12707 raft_consensus.cc:2272] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 77c6c01846a946c49f93ed187dfb4b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.669781 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:09:04.686733 12707 tablet_server.cc:179] TabletServer@127.12.104.195:0 shutting down...
I20260924 15:09:04.724807 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:04.724943 14825 raft_consensus.cc:493] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 79bbea6671054091bc8e3c57ceda1a68)
I20260924 15:09:04.725677 14825 raft_consensus.cc:515] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a4a328946541a9be28ac15366e866d" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39951 } } peers { permanent_uuid: "79bbea6671054091bc8e3c57ceda1a68" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 38665 } } peers { permanent_uuid: "77c6c01846a946c49f93ed187dfb4b11" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 43571 } }
I20260924 15:09:04.728066 14825 leader_election.cc:290] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665), 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571)
I20260924 15:09:04.731706 12707 tablet_replica.cc:333] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d: stopping tablet replica
I20260924 15:09:04.733580 12707 raft_consensus.cc:2243] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.734361 12707 raft_consensus.cc:2272] T 47871e968d7a49dabd43afc59d8fe10e P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 15:09:04.735406 14586 leader_election.cc:336] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 79bbea6671054091bc8e3c57ceda1a68 (127.12.104.193:38665): Network error: Client connection negotiation failed: client connection to 127.12.104.193:38665: connect: Connection refused (error 111)
I20260924 15:09:04.737447 12707 tablet_replica.cc:333] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: stopping tablet replica
I20260924 15:09:04.738291 12707 raft_consensus.cc:2243] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.738956 12707 raft_consensus.cc:2272] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260924 15:09:04.740679 14585 leader_election.cc:336] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 77c6c01846a946c49f93ed187dfb4b11 (127.12.104.194:43571): Network error: Client connection negotiation failed: client connection to 127.12.104.194:43571: connect: Connection refused (error 111)
I20260924 15:09:04.741170 14585 leader_election.cc:304] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a4a328946541a9be28ac15366e866d; no voters: 77c6c01846a946c49f93ed187dfb4b11, 79bbea6671054091bc8e3c57ceda1a68
I20260924 15:09:04.741482 12707 tablet_replica.cc:333] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d: stopping tablet replica
W20260924 15:09:04.741882 14585 raft_consensus.cc:2686] T a2fda22f30124c7eaaaff155e72355f3 P 87a4a328946541a9be28ac15366e866d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 15:09:04.742197 12707 raft_consensus.cc:2243] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:04.743108 12707 raft_consensus.cc:2272] T 4e648c07d8ee4c5bb1062d8d3b6a9bea P 87a4a328946541a9be28ac15366e866d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.789893 12707 tablet_server.cc:196] TabletServer@127.12.104.195:0 shutdown complete.
I20260924 15:09:04.805124 12707 master.cc:562] Master@127.12.104.254:45779 shutting down...
I20260924 15:09:04.836051 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 5 LEADER]: Raft consensus shutting down.
I20260924 15:09:04.837133 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.837656 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 08536d2250d14b76957e5b87dd30a9e7: stopping tablet replica
I20260924 15:09:04.860690 12707 master.cc:584] Master@127.12.104.254:45779 shutdown complete.
I20260924 15:09:04.880927 12707 master.cc:562] Master@127.12.104.253:39543 shutting down...
I20260924 15:09:04.904464 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:04.905014 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:04.905342 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0a97f8d8b90b4e6d910032e7df7cff96: stopping tablet replica
I20260924 15:09:04.934028 14847 raft_consensus.cc:493] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 08536d2250d14b76957e5b87dd30a9e7)
I20260924 15:09:04.934725 14847 raft_consensus.cc:515] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08536d2250d14b76957e5b87dd30a9e7" member_type: VOTER last_known_addr { host: "127.12.104.254" port: 45779 } } peers { permanent_uuid: "0a97f8d8b90b4e6d910032e7df7cff96" member_type: VOTER last_known_addr { host: "127.12.104.253" port: 39543 } } peers { permanent_uuid: "a57037ec14994dd5ad4f06193c9ec648" member_type: VOTER last_known_addr { host: "127.12.104.252" port: 41327 } }
I20260924 15:09:04.937878 14847 leader_election.cc:290] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779), 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543)
W20260924 15:09:04.944556 14307 leader_election.cc:336] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 08536d2250d14b76957e5b87dd30a9e7 (127.12.104.254:45779): Network error: Client connection negotiation failed: client connection to 127.12.104.254:45779: connect: Connection refused (error 111)
W20260924 15:09:04.945230 14304 leader_election.cc:336] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0a97f8d8b90b4e6d910032e7df7cff96 (127.12.104.253:39543): Network error: Client connection negotiation failed: client connection to 127.12.104.253:39543: connect: Connection refused (error 111)
I20260924 15:09:04.945582 14304 leader_election.cc:304] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a57037ec14994dd5ad4f06193c9ec648; no voters: 08536d2250d14b76957e5b87dd30a9e7, 0a97f8d8b90b4e6d910032e7df7cff96
I20260924 15:09:04.946233 14847 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 15:09:04.950666 12707 master.cc:584] Master@127.12.104.253:39543 shutdown complete.
I20260924 15:09:04.969427 12707 master.cc:562] Master@127.12.104.252:41327 shutting down...
I20260924 15:09:05.001537 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:05.002233 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:05.002691 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P a57037ec14994dd5ad4f06193c9ec648: stopping tablet replica
I20260924 15:09:05.026293 12707 master.cc:584] Master@127.12.104.252:41327 shutdown complete.
I20260924 15:09:05.056375 12707 test_util.cc:182] -----------------------------------------------
I20260924 15:09:05.056633 12707 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790262492846473-12707-0
[ FAILED ] TsTabletManagerITest.TestTableStats (34130 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260924 15:09:05.064853 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:42699
I20260924 15:09:05.066612 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:05.074296 14856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.075585 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:05.077529 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:05.077629 14859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:05.078923 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:05.079275 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:05.079507 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262545079489 us; error 0 us; skew 500 ppm
I20260924 15:09:05.080303 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:05.083810 12707 webserver.cc:533] Webserver started at http://127.12.104.254:43825/ using document root <none> and password file <none>
I20260924 15:09:05.084328 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:05.084501 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:05.084739 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:05.086024 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "da9d6db0dc41440a8580c99f3fdb3511"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.091188 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260924 15:09:05.095503 14864 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.096611 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 15:09:05.097029 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "da9d6db0dc41440a8580c99f3fdb3511"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.097422 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:05.115317 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:05.116667 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:05.165616 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:42699
I20260924 15:09:05.165766 14915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:42699 every 8 connection(s)
I20260924 15:09:05.171299 14916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:05.185143 14916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: Bootstrap starting.
I20260924 15:09:05.192358 14916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:05.198060 14916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: No bootstrap required, opened a new log
I20260924 15:09:05.200815 14916 raft_consensus.cc:359] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER }
I20260924 15:09:05.201287 14916 raft_consensus.cc:385] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:05.201488 14916 raft_consensus.cc:740] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9d6db0dc41440a8580c99f3fdb3511, State: Initialized, Role: FOLLOWER
I20260924 15:09:05.202102 14916 consensus_queue.cc:260] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER }
I20260924 15:09:05.202577 14916 raft_consensus.cc:399] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:05.202771 14916 raft_consensus.cc:493] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:05.202982 14916 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:05.207885 14916 raft_consensus.cc:515] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER }
I20260924 15:09:05.208416 14916 leader_election.cc:304] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da9d6db0dc41440a8580c99f3fdb3511; no voters:
I20260924 15:09:05.209707 14916 leader_election.cc:290] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:05.210145 14919 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:05.211733 14919 raft_consensus.cc:697] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 1 LEADER]: Becoming Leader. State: Replica: da9d6db0dc41440a8580c99f3fdb3511, State: Running, Role: LEADER
I20260924 15:09:05.212531 14919 consensus_queue.cc:237] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER }
I20260924 15:09:05.212893 14916 sys_catalog.cc:565] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:05.215942 14920 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da9d6db0dc41440a8580c99f3fdb3511" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER } }
I20260924 15:09:05.215942 14921 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da9d6db0dc41440a8580c99f3fdb3511. Latest consensus state: current_term: 1 leader_uuid: "da9d6db0dc41440a8580c99f3fdb3511" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9d6db0dc41440a8580c99f3fdb3511" member_type: VOTER } }
I20260924 15:09:05.216821 14920 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:05.216842 14921 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:05.225677 14924 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:05.234059 14924 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:05.235314 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:05.244539 14924 catalog_manager.cc:1396] Generated new cluster ID: 5662c0c6d0754273931ed7525fe64b7b
I20260924 15:09:05.244930 14924 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:05.270491 14924 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:05.271987 14924 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:05.288844 14924 catalog_manager.cc:6219] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: Generated new TSK 0
I20260924 15:09:05.289713 14924 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:05.302270 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:05.309824 14938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.311305 14939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.313057 14941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:05.315845 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:05.317143 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:05.317364 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:05.317596 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262545317557 us; error 0 us; skew 500 ppm
I20260924 15:09:05.318208 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:05.321201 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40179/ using document root <none> and password file <none>
I20260924 15:09:05.321765 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:05.321974 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:05.322286 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:05.323544 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "4640d1c91fd44c54aad38112eb1d580a"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.328696 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260924 15:09:05.332796 14947 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.333748 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 15:09:05.334220 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "4640d1c91fd44c54aad38112eb1d580a"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.334640 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:05.350068 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:05.351486 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:05.353421 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:05.356709 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:05.357002 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.357290 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:05.357486 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.411684 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:34713
I20260924 15:09:05.411847 15010 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:34713 every 8 connection(s)
I20260924 15:09:05.416878 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:05.425486 15015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.427660 15016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.429947 15018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:05.429984 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:05.431828 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:05.432122 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:05.432325 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262545432300 us; error 0 us; skew 500 ppm
I20260924 15:09:05.433075 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:05.436357 12707 webserver.cc:533] Webserver started at http://127.12.104.194:36675/ using document root <none> and password file <none>
I20260924 15:09:05.437083 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:05.437398 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:05.437716 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:05.439083 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "2d8721bd213e468eade5572589403216"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.441468 15011 heartbeater.cc:344] Connected to a master server at 127.12.104.254:42699
I20260924 15:09:05.441967 15011 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:05.442925 15011 heartbeater.cc:507] Master 127.12.104.254:42699 requested a full tablet report, sending...
I20260924 15:09:05.445600 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260924 15:09:05.445847 14881 ts_manager.cc:194] Registered new tserver with Master: 4640d1c91fd44c54aad38112eb1d580a (127.12.104.193:34713)
I20260924 15:09:05.447865 14881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38582
I20260924 15:09:05.449990 15023 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.451076 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:05.451366 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "2d8721bd213e468eade5572589403216"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.451664 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:05.474611 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:05.475859 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:05.477547 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:05.480096 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:05.480301 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.480669 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:05.480821 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.526435 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:39909
I20260924 15:09:05.526537 15087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:39909 every 8 connection(s)
I20260924 15:09:05.531615 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:05.539844 15091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.542006 15092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:05.546376 15094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:05.546422 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:05.547003 15088 heartbeater.cc:344] Connected to a master server at 127.12.104.254:42699
I20260924 15:09:05.547677 15088 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:05.547958 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:05.548237 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:05.548442 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262545548420 us; error 0 us; skew 500 ppm
I20260924 15:09:05.549111 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:05.549033 15088 heartbeater.cc:507] Master 127.12.104.254:42699 requested a full tablet report, sending...
I20260924 15:09:05.552608 12707 webserver.cc:533] Webserver started at http://127.12.104.195:35561/ using document root <none> and password file <none>
I20260924 15:09:05.552671 14881 ts_manager.cc:194] Registered new tserver with Master: 2d8721bd213e468eade5572589403216 (127.12.104.194:39909)
I20260924 15:09:05.553438 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:05.553730 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:05.554131 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:05.554605 14881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38596
I20260924 15:09:05.555492 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root/instance:
uuid: "a7939b165fb847bda58dbd10d0640dba"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.561290 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 15:09:05.565730 15099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.567054 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 15:09:05.567495 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root
uuid: "a7939b165fb847bda58dbd10d0640dba"
format_stamp: "Formatted at 2026-09-24 15:09:05 on dist-test-slave-9164"
I20260924 15:09:05.567792 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:05.601660 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:05.603070 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:05.604588 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:05.607239 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:05.607503 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.607793 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:05.607992 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:05.656720 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.195:34141
I20260924 15:09:05.656834 15161 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.195:34141 every 8 connection(s)
I20260924 15:09:05.674275 15162 heartbeater.cc:344] Connected to a master server at 127.12.104.254:42699
I20260924 15:09:05.674768 15162 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:05.675662 15162 heartbeater.cc:507] Master 127.12.104.254:42699 requested a full tablet report, sending...
I20260924 15:09:05.678779 14881 ts_manager.cc:194] Registered new tserver with Master: a7939b165fb847bda58dbd10d0640dba (127.12.104.195:34141)
I20260924 15:09:05.679224 12707 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019065665s
I20260924 15:09:05.680406 14881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38600
I20260924 15:09:05.719094 14881 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260924 15:09:05.720913 14881 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 15:09:05.768720 14975 tablet_service.cc:1511] Processing CreateTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 (DEFAULT_TABLE table=test-table [id=d0f26fec168b4d879b861a8a45b6971b]), partition=
I20260924 15:09:05.770120 14975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2008ef645d8544578d0bbfc92b69a0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:05.774632 15051 tablet_service.cc:1511] Processing CreateTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 (DEFAULT_TABLE table=test-table [id=d0f26fec168b4d879b861a8a45b6971b]), partition=
I20260924 15:09:05.775861 15051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2008ef645d8544578d0bbfc92b69a0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:05.776543 15127 tablet_service.cc:1511] Processing CreateTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 (DEFAULT_TABLE table=test-table [id=d0f26fec168b4d879b861a8a45b6971b]), partition=
I20260924 15:09:05.778021 15127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2008ef645d8544578d0bbfc92b69a0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:05.790964 15178 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Bootstrap starting.
I20260924 15:09:05.792748 15179 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Bootstrap starting.
I20260924 15:09:05.799724 15178 tablet_bootstrap.cc:654] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:05.800246 15180 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Bootstrap starting.
I20260924 15:09:05.801275 15179 tablet_bootstrap.cc:654] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:05.809233 15180 tablet_bootstrap.cc:654] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:05.810406 15179 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: No bootstrap required, opened a new log
I20260924 15:09:05.811139 15179 ts_tablet_manager.cc:1403] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.014s
I20260924 15:09:05.810487 15178 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: No bootstrap required, opened a new log
I20260924 15:09:05.812999 15178 ts_tablet_manager.cc:1403] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.008s
I20260924 15:09:05.814749 15179 raft_consensus.cc:359] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.815203 15180 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: No bootstrap required, opened a new log
I20260924 15:09:05.815546 15179 raft_consensus.cc:385] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:05.816000 15179 raft_consensus.cc:740] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d8721bd213e468eade5572589403216, State: Initialized, Role: FOLLOWER
I20260924 15:09:05.816016 15180 ts_tablet_manager.cc:1403] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260924 15:09:05.816195 15178 raft_consensus.cc:359] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.817181 15178 raft_consensus.cc:385] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:05.817111 15179 consensus_queue.cc:260] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.817721 15178 raft_consensus.cc:740] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4640d1c91fd44c54aad38112eb1d580a, State: Initialized, Role: FOLLOWER
I20260924 15:09:05.818617 15178 consensus_queue.cc:260] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.820053 15180 raft_consensus.cc:359] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.821130 15180 raft_consensus.cc:385] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:05.821550 15180 raft_consensus.cc:740] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7939b165fb847bda58dbd10d0640dba, State: Initialized, Role: FOLLOWER
I20260924 15:09:05.822679 15180 consensus_queue.cc:260] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.826223 15088 heartbeater.cc:499] Master 127.12.104.254:42699 was elected leader, sending a full tablet report...
I20260924 15:09:05.826419 15179 ts_tablet_manager.cc:1434] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
W20260924 15:09:05.826769 15179 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260924 15:09:05.828068 15011 heartbeater.cc:499] Master 127.12.104.254:42699 was elected leader, sending a full tablet report...
I20260924 15:09:05.829193 15178 ts_tablet_manager.cc:1434] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260924 15:09:05.829591 15178 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260924 15:09:05.831151 15162 heartbeater.cc:499] Master 127.12.104.254:42699 was elected leader, sending a full tablet report...
I20260924 15:09:05.832908 15180 ts_tablet_manager.cc:1434] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Time spent starting tablet: real 0.016s user 0.008s sys 0.003s
W20260924 15:09:05.833254 15180 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260924 15:09:05.854168 15185 raft_consensus.cc:493] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:09:05.854753 15185 raft_consensus.cc:515] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.856930 15185 leader_election.cc:290] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7939b165fb847bda58dbd10d0640dba (127.12.104.195:34141), 2d8721bd213e468eade5572589403216 (127.12.104.194:39909)
I20260924 15:09:05.866254 15137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2008ef645d8544578d0bbfc92b69a0c5" candidate_uuid: "4640d1c91fd44c54aad38112eb1d580a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7939b165fb847bda58dbd10d0640dba" is_pre_election: true
I20260924 15:09:05.867185 15137 raft_consensus.cc:2468] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4640d1c91fd44c54aad38112eb1d580a in term 0.
I20260924 15:09:05.868748 14950 leader_election.cc:304] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4640d1c91fd44c54aad38112eb1d580a, a7939b165fb847bda58dbd10d0640dba; no voters:
I20260924 15:09:05.869812 15185 raft_consensus.cc:2804] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:09:05.869902 15061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2008ef645d8544578d0bbfc92b69a0c5" candidate_uuid: "4640d1c91fd44c54aad38112eb1d580a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8721bd213e468eade5572589403216" is_pre_election: true
I20260924 15:09:05.870342 15185 raft_consensus.cc:493] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:09:05.870744 15185 raft_consensus.cc:3060] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:05.870759 15061 raft_consensus.cc:2468] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4640d1c91fd44c54aad38112eb1d580a in term 0.
I20260924 15:09:05.876974 15185 raft_consensus.cc:515] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.879288 15185 leader_election.cc:290] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [CANDIDATE]: Term 1 election: Requested vote from peers a7939b165fb847bda58dbd10d0640dba (127.12.104.195:34141), 2d8721bd213e468eade5572589403216 (127.12.104.194:39909)
I20260924 15:09:05.880295 15137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2008ef645d8544578d0bbfc92b69a0c5" candidate_uuid: "4640d1c91fd44c54aad38112eb1d580a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7939b165fb847bda58dbd10d0640dba"
I20260924 15:09:05.880543 15061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2008ef645d8544578d0bbfc92b69a0c5" candidate_uuid: "4640d1c91fd44c54aad38112eb1d580a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8721bd213e468eade5572589403216"
I20260924 15:09:05.880939 15137 raft_consensus.cc:3060] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:05.881055 15061 raft_consensus.cc:3060] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:05.888928 15137 raft_consensus.cc:2468] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4640d1c91fd44c54aad38112eb1d580a in term 1.
I20260924 15:09:05.888928 15061 raft_consensus.cc:2468] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4640d1c91fd44c54aad38112eb1d580a in term 1.
I20260924 15:09:05.890576 14950 leader_election.cc:304] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4640d1c91fd44c54aad38112eb1d580a, a7939b165fb847bda58dbd10d0640dba; no voters:
I20260924 15:09:05.891494 15185 raft_consensus.cc:2804] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:05.892721 15185 raft_consensus.cc:697] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 LEADER]: Becoming Leader. State: Replica: 4640d1c91fd44c54aad38112eb1d580a, State: Running, Role: LEADER
I20260924 15:09:05.893570 15185 consensus_queue.cc:237] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:05.901801 14879 catalog_manager.cc:5838] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4640d1c91fd44c54aad38112eb1d580a (127.12.104.193). New cstate: current_term: 1 leader_uuid: "4640d1c91fd44c54aad38112eb1d580a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } health_report { overall_health: UNKNOWN } } }
W20260924 15:09:05.910946 15163 tablet.cc:2433] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 15:09:05.943042 15051 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:41816
I20260924 15:09:05.944862 15191 tablet_replica.cc:333] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: stopping tablet replica
I20260924 15:09:05.945452 15191 raft_consensus.cc:2243] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:05.945837 15191 raft_consensus.cc:2272] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:05.948077 15191 ts_tablet_manager.cc:1916] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 15:09:05.959551 15191 ts_tablet_manager.cc:1929] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260924 15:09:05.959919 15191 log.cc:1199] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root/wals/2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:05.964632 15192 ts_tablet_manager.cc:933] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Initiating tablet copy from peer 4640d1c91fd44c54aad38112eb1d580a (127.12.104.193:34713)
I20260924 15:09:05.966518 15192 tablet_copy_client.cc:323] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet copy: Beginning tablet copy session from remote peer at address 127.12.104.193:34713
I20260924 15:09:05.977152 14996 tablet_copy_service.cc:140] P 4640d1c91fd44c54aad38112eb1d580a: Received BeginTabletCopySession request for tablet 2008ef645d8544578d0bbfc92b69a0c5 from peer 2d8721bd213e468eade5572589403216 ({username='slave'} at 127.0.0.1:38148)
I20260924 15:09:05.977821 14996 tablet_copy_service.cc:161] P 4640d1c91fd44c54aad38112eb1d580a: Beginning new tablet copy session on tablet 2008ef645d8544578d0bbfc92b69a0c5 from peer 2d8721bd213e468eade5572589403216 at {username='slave'} at 127.0.0.1:38148: session id = 2d8721bd213e468eade5572589403216-2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:05.983803 14996 tablet_copy_source_session.cc:215] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 15:09:05.986958 15192 ts_tablet_manager.cc:1916] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260924 15:09:05.998397 15192 ts_tablet_manager.cc:1929] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260924 15:09:05.999183 15192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2008ef645d8544578d0bbfc92b69a0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:06.005236 15192 tablet_copy_client.cc:806] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet copy: Starting download of 0 data blocks...
I20260924 15:09:06.005898 15192 tablet_copy_client.cc:670] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet copy: Starting download of 1 WAL segments...
I20260924 15:09:06.008163 14879 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38622:
table { table_id: "d0f26fec168b4d879b861a8a45b6971b" table_name: "test-table" }
I20260924 15:09:06.008811 14879 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38622:
table { table_id: "d0f26fec168b4d879b861a8a45b6971b" table_name: "test-table" }
I20260924 15:09:06.010545 15192 tablet_copy_client.cc:538] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 15:09:06.019140 15192 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Bootstrap starting.
I20260924 15:09:06.020051 14879 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: Sending DeleteTablet for 3 replicas of tablet 2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:06.022262 14975 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 15:09:06 UTC) from {username='slave'} at 127.0.0.1:38138
I20260924 15:09:06.022637 15127 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 15:09:06 UTC) from {username='slave'} at 127.0.0.1:49896
I20260924 15:09:06.023716 15051 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 15:09:06 UTC) from {username='slave'} at 127.0.0.1:41804
I20260924 15:09:06.026381 15194 tablet_replica.cc:333] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: stopping tablet replica
I20260924 15:09:06.027346 15195 tablet_replica.cc:333] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: stopping tablet replica
I20260924 15:09:06.027390 15194 raft_consensus.cc:2243] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:06.028298 15195 raft_consensus.cc:2243] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:06.028446 15194 pending_rounds.cc:70] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Trying to abort 1 pending ops.
W20260924 15:09:06.028535 14866 catalog_manager.cc:5043] TS 2d8721bd213e468eade5572589403216 (127.12.104.194:39909): delete failed for tablet 2008ef645d8544578d0bbfc92b69a0c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2008ef645d8544578d0bbfc92b69a0c5 already in progress: copying tablet
I20260924 15:09:06.028914 15194 pending_rounds.cc:77] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332915387987488768 op_type: NO_OP noop_request { }
I20260924 15:09:06.029554 15194 raft_consensus.cc:2889] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 15:09:06.029603 15195 raft_consensus.cc:2272] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:06.030175 15194 raft_consensus.cc:2272] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:06.032868 15195 ts_tablet_manager.cc:1916] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 15:09:06.033494 15194 ts_tablet_manager.cc:1916] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 15:09:06.037317 15192 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260924 15:09:06.038502 15192 tablet_bootstrap.cc:492] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Bootstrap complete.
I20260924 15:09:06.039265 15192 ts_tablet_manager.cc:1403] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.009s
I20260924 15:09:06.042577 15192 raft_consensus.cc:359] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:06.043733 15192 raft_consensus.cc:740] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d8721bd213e468eade5572589403216, State: Initialized, Role: FOLLOWER
I20260924 15:09:06.044615 15192 consensus_queue.cc:260] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4640d1c91fd44c54aad38112eb1d580a" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 34713 } } peers { permanent_uuid: "a7939b165fb847bda58dbd10d0640dba" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 34141 } } peers { permanent_uuid: "2d8721bd213e468eade5572589403216" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 39909 } }
I20260924 15:09:06.045898 15194 ts_tablet_manager.cc:1929] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 15:09:06.046335 15194 log.cc:1199] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-0-root/wals/2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:06.046775 15192 ts_tablet_manager.cc:1434] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260924 15:09:06.047101 15192 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260924 15:09:06.047240 15194 ts_tablet_manager.cc:1950] T 2008ef645d8544578d0bbfc92b69a0c5 P 4640d1c91fd44c54aad38112eb1d580a: Deleting consensus metadata
I20260924 15:09:06.048919 14868 catalog_manager.cc:5063] TS 4640d1c91fd44c54aad38112eb1d580a (127.12.104.193:34713): tablet 2008ef645d8544578d0bbfc92b69a0c5 (table test-table [id=d0f26fec168b4d879b861a8a45b6971b]) successfully deleted
I20260924 15:09:06.050908 15195 ts_tablet_manager.cc:1929] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260924 15:09:06.051357 15195 log.cc:1199] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-2-root/wals/2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:06.052276 15195 ts_tablet_manager.cc:1950] T 2008ef645d8544578d0bbfc92b69a0c5 P a7939b165fb847bda58dbd10d0640dba: Deleting consensus metadata
I20260924 15:09:06.052975 15051 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 15:09:06 UTC) from {username='slave'} at 127.0.0.1:41804
W20260924 15:09:06.054739 14866 catalog_manager.cc:5043] TS 2d8721bd213e468eade5572589403216 (127.12.104.194:39909): delete failed for tablet 2008ef645d8544578d0bbfc92b69a0c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2008ef645d8544578d0bbfc92b69a0c5 already in progress: copying tablet
I20260924 15:09:06.055748 14867 catalog_manager.cc:5063] TS a7939b165fb847bda58dbd10d0640dba (127.12.104.195:34141): tablet 2008ef645d8544578d0bbfc92b69a0c5 (table test-table [id=d0f26fec168b4d879b861a8a45b6971b]) successfully deleted
I20260924 15:09:06.099610 14996 tablet_copy_service.cc:342] P 4640d1c91fd44c54aad38112eb1d580a: Request end of tablet copy session 2d8721bd213e468eade5572589403216-2008ef645d8544578d0bbfc92b69a0c5 received from {username='slave'} at 127.0.0.1:38148
I20260924 15:09:06.100121 14996 tablet_copy_service.cc:434] P 4640d1c91fd44c54aad38112eb1d580a: ending tablet copy session 2d8721bd213e468eade5572589403216-2008ef645d8544578d0bbfc92b69a0c5 on tablet 2008ef645d8544578d0bbfc92b69a0c5 with peer 2d8721bd213e468eade5572589403216
I20260924 15:09:06.103885 15051 tablet_service.cc:1558] Processing DeleteTablet for tablet 2008ef645d8544578d0bbfc92b69a0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 15:09:06 UTC) from {username='slave'} at 127.0.0.1:41804
I20260924 15:09:06.104717 15191 tablet_replica.cc:333] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: stopping tablet replica
I20260924 15:09:06.105335 15191 raft_consensus.cc:2243] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:06.105734 15191 pending_rounds.cc:70] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Trying to abort 1 pending ops.
I20260924 15:09:06.105993 15191 pending_rounds.cc:77] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332915387987488768 op_type: NO_OP noop_request { }
I20260924 15:09:06.106375 15191 raft_consensus.cc:2889] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260924 15:09:06.106729 15191 raft_consensus.cc:2272] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:06.108942 15191 ts_tablet_manager.cc:1916] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 15:09:06.119904 15191 ts_tablet_manager.cc:1929] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260924 15:09:06.120296 15191 log.cc:1199] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790262492846473-12707-0/minicluster-data/ts-1-root/wals/2008ef645d8544578d0bbfc92b69a0c5
I20260924 15:09:06.120915 15191 ts_tablet_manager.cc:1950] T 2008ef645d8544578d0bbfc92b69a0c5 P 2d8721bd213e468eade5572589403216: Deleting consensus metadata
I20260924 15:09:06.123585 14866 catalog_manager.cc:5063] TS 2d8721bd213e468eade5572589403216 (127.12.104.194:39909): tablet 2008ef645d8544578d0bbfc92b69a0c5 (table test-table [id=d0f26fec168b4d879b861a8a45b6971b]) successfully deleted
I20260924 15:09:06.159229 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:06.183351 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:06.202482 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:06.212615 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:09:06.231812 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:06.249492 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:09:06.261483 12707 tablet_server.cc:179] TabletServer@127.12.104.195:0 shutting down...
I20260924 15:09:06.283236 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:06.301160 12707 tablet_server.cc:196] TabletServer@127.12.104.195:0 shutdown complete.
I20260924 15:09:06.311228 12707 master.cc:562] Master@127.12.104.254:42699 shutting down...
I20260924 15:09:06.328115 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:06.328732 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:06.329099 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P da9d6db0dc41440a8580c99f3fdb3511: stopping tablet replica
I20260924 15:09:06.348801 12707 master.cc:584] Master@127.12.104.254:42699 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1317 ms)
[----------] 4 tests from TsTabletManagerITest (42042 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260924 15:09:06.381767 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:37259
I20260924 15:09:06.383073 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:06.388780 15201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:06.389878 15202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:06.392550 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:06.392904 15204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:06.393857 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:06.394052 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:06.394170 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262546394159 us; error 0 us; skew 500 ppm
I20260924 15:09:06.394761 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:06.397228 12707 webserver.cc:533] Webserver started at http://127.12.104.254:41431/ using document root <none> and password file <none>
I20260924 15:09:06.397820 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:06.397994 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:06.398226 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:06.399574 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174"
format_stamp: "Formatted at 2026-09-24 15:09:06 on dist-test-slave-9164"
I20260924 15:09:06.404949 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 15:09:06.410099 15209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:06.411166 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:06.411499 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174"
format_stamp: "Formatted at 2026-09-24 15:09:06 on dist-test-slave-9164"
I20260924 15:09:06.411796 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:06.429421 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:06.430729 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:06.477505 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:37259
I20260924 15:09:06.477634 15262 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:37259 every 8 connection(s)
I20260924 15:09:06.483049 15263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:06.495242 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174: Bootstrap starting.
I20260924 15:09:06.500566 15263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:06.505257 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174: No bootstrap required, opened a new log
I20260924 15:09:06.507776 15263 raft_consensus.cc:359] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER }
I20260924 15:09:06.508227 15263 raft_consensus.cc:385] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:06.508530 15263 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bbe9d0628448ce9bbb0b6bd1c1e174, State: Initialized, Role: FOLLOWER
I20260924 15:09:06.509234 15263 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER }
I20260924 15:09:06.509723 15263 raft_consensus.cc:399] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:06.510064 15263 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:06.510444 15263 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:06.515822 15263 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER }
I20260924 15:09:06.516400 15263 leader_election.cc:304] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9bbe9d0628448ce9bbb0b6bd1c1e174; no voters:
I20260924 15:09:06.517764 15263 leader_election.cc:290] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:06.518203 15266 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:06.519680 15266 raft_consensus.cc:697] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 1 LEADER]: Becoming Leader. State: Replica: a9bbe9d0628448ce9bbb0b6bd1c1e174, State: Running, Role: LEADER
I20260924 15:09:06.520382 15266 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER }
I20260924 15:09:06.520841 15263 sys_catalog.cc:565] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:06.525503 15268 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bbe9d0628448ce9bbb0b6bd1c1e174. Latest consensus state: current_term: 1 leader_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER } }
I20260924 15:09:06.526225 15268 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:06.526473 15267 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bbe9d0628448ce9bbb0b6bd1c1e174" member_type: VOTER } }
I20260924 15:09:06.527036 15267 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:06.528393 15274 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:06.533384 15274 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:06.537307 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:06.543820 15274 catalog_manager.cc:1396] Generated new cluster ID: af00d89e34c943a588af4c327f898062
I20260924 15:09:06.544176 15274 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:06.568480 15274 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:06.569772 15274 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:06.595909 15274 catalog_manager.cc:6219] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174: Generated new TSK 0
I20260924 15:09:06.596570 15274 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:06.605144 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:06.612138 15284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:06.613552 15285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:06.615651 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:06.615837 15287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:06.616894 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:06.617156 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:06.617359 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262546617327 us; error 0 us; skew 500 ppm
I20260924 15:09:06.617908 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:06.620445 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40465/ using document root <none> and password file <none>
I20260924 15:09:06.620975 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:06.621196 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:06.621476 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:06.622731 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "da2abf0b175d48668617774de9d3e8e9"
format_stamp: "Formatted at 2026-09-24 15:09:06 on dist-test-slave-9164"
I20260924 15:09:06.627574 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 15:09:06.631340 15292 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:06.632179 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260924 15:09:06.632510 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "da2abf0b175d48668617774de9d3e8e9"
format_stamp: "Formatted at 2026-09-24 15:09:06 on dist-test-slave-9164"
I20260924 15:09:06.632807 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:06.656203 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:06.657476 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:06.662644 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:06.663071 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260924 15:09:06.663465 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:06.663867 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:06.717617 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:39203
I20260924 15:09:06.717712 15363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:39203 every 8 connection(s)
I20260924 15:09:06.722676 15364 heartbeater.cc:344] Connected to a master server at 127.12.104.254:37259
I20260924 15:09:06.723049 15364 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:06.723773 15364 heartbeater.cc:507] Master 127.12.104.254:37259 requested a full tablet report, sending...
I20260924 15:09:06.725728 15228 ts_manager.cc:194] Registered new tserver with Master: da2abf0b175d48668617774de9d3e8e9 (127.12.104.193:39203)
I20260924 15:09:06.725849 12707 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004892611s
I20260924 15:09:06.727569 15228 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38308
I20260924 15:09:06.761610 15328 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260924 15:09:06.763298 15328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:06.776540 15374 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Bootstrap starting.
I20260924 15:09:06.781665 15374 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:06.786019 15374 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: No bootstrap required, opened a new log
I20260924 15:09:06.786520 15374 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260924 15:09:06.788563 15374 raft_consensus.cc:359] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:06.789003 15374 raft_consensus.cc:385] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:06.789237 15374 raft_consensus.cc:740] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2abf0b175d48668617774de9d3e8e9, State: Initialized, Role: FOLLOWER
I20260924 15:09:06.789850 15374 consensus_queue.cc:260] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:06.790400 15374 raft_consensus.cc:399] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:06.790664 15374 raft_consensus.cc:493] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:06.790934 15374 raft_consensus.cc:3060] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:06.795742 15374 raft_consensus.cc:515] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:06.796339 15374 leader_election.cc:304] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2abf0b175d48668617774de9d3e8e9; no voters:
I20260924 15:09:06.797683 15374 leader_election.cc:290] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:06.798091 15376 raft_consensus.cc:2804] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:06.799552 15376 raft_consensus.cc:697] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 LEADER]: Becoming Leader. State: Replica: da2abf0b175d48668617774de9d3e8e9, State: Running, Role: LEADER
I20260924 15:09:06.799780 15364 heartbeater.cc:499] Master 127.12.104.254:37259 was elected leader, sending a full tablet report...
I20260924 15:09:06.800360 15376 consensus_queue.cc:237] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:06.800779 15374 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260924 15:09:06.805404 15377 tablet_replica.cc:442] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2abf0b175d48668617774de9d3e8e9" committed_config { opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } } }
I20260924 15:09:06.805646 15378 tablet_replica.cc:442] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: TxnStatusTablet state changed. Reason: New leader da2abf0b175d48668617774de9d3e8e9. Latest consensus state: current_term: 1 leader_uuid: "da2abf0b175d48668617774de9d3e8e9" committed_config { opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } } }
I20260924 15:09:06.806138 15377 tablet_replica.cc:445] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:06.806542 15378 tablet_replica.cc:445] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: This TxnStatusTablet replica's current role is: LEADER
W20260924 15:09:06.808624 15228 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:06.810976 15381 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:06.811460 15381 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260924 15:09:06.813731 15228 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:06.851263 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:06.890606 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:06.891518 12707 tablet_replica.cc:333] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: stopping tablet replica
I20260924 15:09:06.892180 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:06.892719 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:06.915215 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:06.931216 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:06.938454 15384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:06.938933 15385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:06.940389 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:06.941972 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:06.943053 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:06.943284 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:06.943462 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262546943445 us; error 0 us; skew 500 ppm
I20260924 15:09:06.944038 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:06.947101 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40465/ using document root <none> and password file <none>
I20260924 15:09:06.947628 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:06.947834 12707 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260924 15:09:06.951889 12707 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260924 15:09:06.955226 15392 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:06.956189 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 15:09:06.956530 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "da2abf0b175d48668617774de9d3e8e9"
format_stamp: "Formatted at 2026-09-24 15:09:06 on dist-test-slave-9164"
I20260924 15:09:06.956842 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:06.973688 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:06.975035 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:06.984184 15401 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20260924 15:09:06.993849 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 15:09:06.994196 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.011s user 0.002s sys 0.000s
I20260924 15:09:06.994558 12707 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260924 15:09:07.003073 12707 ts_tablet_manager.cc:616] Registered 1 tablets
I20260924 15:09:07.003446 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.009s user 0.008s sys 0.000s
I20260924 15:09:07.003487 15401 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Bootstrap starting.
I20260924 15:09:07.049136 15401 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 15:09:07.050385 15401 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Bootstrap complete.
I20260924 15:09:07.051301 15401 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Time spent bootstrapping tablet: real 0.048s user 0.037s sys 0.008s
I20260924 15:09:07.054529 15401 raft_consensus.cc:359] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:07.055264 15401 raft_consensus.cc:740] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2abf0b175d48668617774de9d3e8e9, State: Initialized, Role: FOLLOWER
I20260924 15:09:07.055992 15401 consensus_queue.cc:260] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:07.056540 15401 raft_consensus.cc:399] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:07.056839 15401 raft_consensus.cc:493] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:07.057188 15401 raft_consensus.cc:3060] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:09:07.064723 15401 raft_consensus.cc:515] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:07.065654 15401 leader_election.cc:304] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2abf0b175d48668617774de9d3e8e9; no voters:
I20260924 15:09:07.068859 15401 leader_election.cc:290] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 15:09:07.069159 15436 raft_consensus.cc:2804] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 15:09:07.080540 15401 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9: Time spent starting tablet: real 0.029s user 0.014s sys 0.011s
I20260924 15:09:07.084082 15436 raft_consensus.cc:697] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 2 LEADER]: Becoming Leader. State: Replica: da2abf0b175d48668617774de9d3e8e9, State: Running, Role: LEADER
I20260924 15:09:07.085222 15436 consensus_queue.cc:237] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } }
I20260924 15:09:07.088774 15439 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "da2abf0b175d48668617774de9d3e8e9" committed_config { opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } } }
I20260924 15:09:07.089470 15439 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:07.089618 15440 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader da2abf0b175d48668617774de9d3e8e9. Latest consensus state: current_term: 2 leader_uuid: "da2abf0b175d48668617774de9d3e8e9" committed_config { opid_index: -1 peers { permanent_uuid: "da2abf0b175d48668617774de9d3e8e9" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 39203 } } }
I20260924 15:09:07.090200 15440 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:07.101495 15447 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:07.102026 15447 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260924 15:09:07.127034 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:39203
I20260924 15:09:07.127344 15470 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:39203 every 8 connection(s)
I20260924 15:09:07.135146 15471 heartbeater.cc:344] Connected to a master server at 127.12.104.254:37259
I20260924 15:09:07.135641 15471 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:07.137686 15471 heartbeater.cc:507] Master 127.12.104.254:37259 requested a full tablet report, sending...
I20260924 15:09:07.138463 12707 tablet_server.cc:179] TabletServer@127.12.104.193:39203 shutting down...
I20260924 15:09:07.142568 15228 ts_manager.cc:194] Re-registered known tserver with Master: da2abf0b175d48668617774de9d3e8e9 (127.12.104.193:39203)
W20260924 15:09:07.143329 15228 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:07.145530 15228 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38340
I20260924 15:09:07.149165 15471 heartbeater.cc:499] Master 127.12.104.254:37259 was elected leader, sending a full tablet report...
W20260924 15:09:07.154388 15228 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:07.179920 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:07.180950 12707 tablet_replica.cc:333] stopping tablet replica
I20260924 15:09:07.181521 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 2 LEADER]: Raft consensus shutting down.
I20260924 15:09:07.182019 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P da2abf0b175d48668617774de9d3e8e9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:07.203763 12707 tablet_server.cc:196] TabletServer@127.12.104.193:39203 shutdown complete.
I20260924 15:09:07.217362 12707 master.cc:562] Master@127.12.104.254:37259 shutting down...
I20260924 15:09:07.235991 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:07.236680 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:07.237076 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P a9bbe9d0628448ce9bbb0b6bd1c1e174: stopping tablet replica
I20260924 15:09:07.258055 12707 master.cc:584] Master@127.12.104.254:37259 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (902 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260924 15:09:07.285209 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:36199
I20260924 15:09:07.286600 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:07.292661 15477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:07.294394 15479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:07.297366 15481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:07.298377 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:07.299185 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:07.299400 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:07.299595 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262547299581 us; error 0 us; skew 500 ppm
I20260924 15:09:07.300190 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:07.303057 12707 webserver.cc:533] Webserver started at http://127.12.104.254:46805/ using document root <none> and password file <none>
I20260924 15:09:07.303588 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:07.303789 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:07.304083 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:07.305212 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "3f419044c09d46cf9c3248e0cbac7414"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.310214 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260924 15:09:07.314028 15486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.314910 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:07.315218 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "3f419044c09d46cf9c3248e0cbac7414"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.315527 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:07.329403 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:07.330637 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:07.384557 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:36199
I20260924 15:09:07.384656 15538 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:36199 every 8 connection(s)
I20260924 15:09:07.389318 15539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:07.400460 15539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414: Bootstrap starting.
I20260924 15:09:07.405148 15539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:07.409992 15539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414: No bootstrap required, opened a new log
I20260924 15:09:07.413200 15539 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER }
I20260924 15:09:07.413652 15539 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:07.413920 15539 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f419044c09d46cf9c3248e0cbac7414, State: Initialized, Role: FOLLOWER
I20260924 15:09:07.414671 15539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER }
I20260924 15:09:07.415177 15539 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:07.415429 15539 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:07.415727 15539 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:07.420861 15539 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER }
I20260924 15:09:07.421459 15539 leader_election.cc:304] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f419044c09d46cf9c3248e0cbac7414; no voters:
I20260924 15:09:07.423029 15539 leader_election.cc:290] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:07.423398 15542 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:07.425123 15542 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 1 LEADER]: Becoming Leader. State: Replica: 3f419044c09d46cf9c3248e0cbac7414, State: Running, Role: LEADER
I20260924 15:09:07.425895 15542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER }
I20260924 15:09:07.426640 15539 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:07.429081 15543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f419044c09d46cf9c3248e0cbac7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER } }
I20260924 15:09:07.429142 15544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f419044c09d46cf9c3248e0cbac7414. Latest consensus state: current_term: 1 leader_uuid: "3f419044c09d46cf9c3248e0cbac7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f419044c09d46cf9c3248e0cbac7414" member_type: VOTER } }
I20260924 15:09:07.429806 15543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:07.429869 15544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:07.432025 15547 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:07.441498 15547 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:07.457482 15547 catalog_manager.cc:1396] Generated new cluster ID: ac12a7a679cd45eab324c7ae36269347
I20260924 15:09:07.457906 15547 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:07.459579 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:07.485440 15547 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:07.487251 15547 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:07.509168 15547 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414: Generated new TSK 0
I20260924 15:09:07.509850 15547 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:07.526572 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:07.533931 15560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:07.539253 15561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:07.540439 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:07.540725 15563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:07.541945 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:07.542222 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:07.542460 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262547542441 us; error 0 us; skew 500 ppm
I20260924 15:09:07.543167 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:07.546132 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40493/ using document root <none> and password file <none>
I20260924 15:09:07.546826 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:07.547089 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:07.547435 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:07.548966 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "824a763e0bcf41b6bba4e64c000679af"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.555522 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260924 15:09:07.560246 15568 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.561290 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 15:09:07.561668 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "824a763e0bcf41b6bba4e64c000679af"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.562053 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:07.598727 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:07.600142 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:07.612782 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:07.613113 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.613440 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:07.613643 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.687424 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:40463
I20260924 15:09:07.687496 15645 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:40463 every 8 connection(s)
I20260924 15:09:07.693747 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260924 15:09:07.693804 15646 heartbeater.cc:344] Connected to a master server at 127.12.104.254:36199
I20260924 15:09:07.694465 15646 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:07.695243 15646 heartbeater.cc:507] Master 127.12.104.254:36199 requested a full tablet report, sending...
I20260924 15:09:07.697268 15503 ts_manager.cc:194] Registered new tserver with Master: 824a763e0bcf41b6bba4e64c000679af (127.12.104.193:40463)
I20260924 15:09:07.699081 15503 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39170
W20260924 15:09:07.702792 15648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:07.704716 15649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:07.708034 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:07.708575 15651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:07.709628 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:07.709923 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:07.710146 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262547710127 us; error 0 us; skew 500 ppm
I20260924 15:09:07.710906 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:07.714018 12707 webserver.cc:533] Webserver started at http://127.12.104.194:39221/ using document root <none> and password file <none>
I20260924 15:09:07.714731 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:07.714983 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:07.715313 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:07.716862 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "43e30527ffcd4c03ad145731b87ec8db"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.723419 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260924 15:09:07.728152 15656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.729184 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 15:09:07.729570 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "43e30527ffcd4c03ad145731b87ec8db"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:07.729947 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:07.741984 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:07.743618 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:07.750634 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:07.750991 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.751307 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:07.751543 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:07.831444 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:44387
I20260924 15:09:07.831543 15725 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:44387 every 8 connection(s)
I20260924 15:09:07.837906 15726 heartbeater.cc:344] Connected to a master server at 127.12.104.254:36199
I20260924 15:09:07.838407 15726 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:07.839414 15726 heartbeater.cc:507] Master 127.12.104.254:36199 requested a full tablet report, sending...
I20260924 15:09:07.841764 15503 ts_manager.cc:194] Registered new tserver with Master: 43e30527ffcd4c03ad145731b87ec8db (127.12.104.194:44387)
I20260924 15:09:07.842497 12707 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.006181531s
I20260924 15:09:07.843679 15503 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39184
I20260924 15:09:07.881742 15610 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260924 15:09:07.883246 15610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:07.896005 15736 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Bootstrap starting.
I20260924 15:09:07.901804 15736 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:07.906864 15736 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: No bootstrap required, opened a new log
I20260924 15:09:07.907348 15736 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260924 15:09:07.909806 15736 raft_consensus.cc:359] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:07.910223 15736 raft_consensus.cc:385] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:07.910537 15736 raft_consensus.cc:740] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824a763e0bcf41b6bba4e64c000679af, State: Initialized, Role: FOLLOWER
I20260924 15:09:07.911180 15736 consensus_queue.cc:260] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:07.911687 15736 raft_consensus.cc:399] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:07.911937 15736 raft_consensus.cc:493] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:07.912222 15736 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:07.917387 15736 raft_consensus.cc:515] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:07.918004 15736 leader_election.cc:304] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824a763e0bcf41b6bba4e64c000679af; no voters:
I20260924 15:09:07.919298 15736 leader_election.cc:290] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:07.919700 15738 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:07.921366 15646 heartbeater.cc:499] Master 127.12.104.254:36199 was elected leader, sending a full tablet report...
I20260924 15:09:07.922227 15736 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260924 15:09:07.922750 15738 raft_consensus.cc:697] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 LEADER]: Becoming Leader. State: Replica: 824a763e0bcf41b6bba4e64c000679af, State: Running, Role: LEADER
I20260924 15:09:07.923520 15738 consensus_queue.cc:237] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:07.928089 15739 tablet_replica.cc:442] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "824a763e0bcf41b6bba4e64c000679af" committed_config { opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } } }
I20260924 15:09:07.928781 15739 tablet_replica.cc:445] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:07.930684 15740 tablet_replica.cc:442] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: TxnStatusTablet state changed. Reason: New leader 824a763e0bcf41b6bba4e64c000679af. Latest consensus state: current_term: 1 leader_uuid: "824a763e0bcf41b6bba4e64c000679af" committed_config { opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } } }
I20260924 15:09:07.931533 15740 tablet_replica.cc:445] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: This TxnStatusTablet replica's current role is: LEADER
W20260924 15:09:07.931845 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:07.940001 15743 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:07.940518 15743 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260924 15:09:07.941740 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:07.995023 15747 ts_tablet_manager.cc:933] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Initiating tablet copy from peer 824a763e0bcf41b6bba4e64c000679af (127.12.104.193:40463)
I20260924 15:09:07.996698 15747 tablet_copy_client.cc:323] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: tablet copy: Beginning tablet copy session from remote peer at address 127.12.104.193:40463
I20260924 15:09:08.007148 15630 tablet_copy_service.cc:140] P 824a763e0bcf41b6bba4e64c000679af: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer 43e30527ffcd4c03ad145731b87ec8db ({username='slave'} at 127.0.0.1:38560)
I20260924 15:09:08.007869 15630 tablet_copy_service.cc:161] P 824a763e0bcf41b6bba4e64c000679af: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer 43e30527ffcd4c03ad145731b87ec8db at {username='slave'} at 127.0.0.1:38560: session id = 43e30527ffcd4c03ad145731b87ec8db-11111111111111111111111111111111
I20260924 15:09:08.016247 15630 tablet_copy_source_session.cc:215] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 15:09:08.019881 15747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:08.032140 15747 tablet_copy_client.cc:806] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: tablet copy: Starting download of 0 data blocks...
I20260924 15:09:08.032737 15747 tablet_copy_client.cc:670] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: tablet copy: Starting download of 1 WAL segments...
I20260924 15:09:08.036891 15747 tablet_copy_client.cc:538] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 15:09:08.043520 15747 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Bootstrap starting.
I20260924 15:09:08.066980 15747 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 15:09:08.067809 15747 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Bootstrap complete.
I20260924 15:09:08.068410 15747 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Time spent bootstrapping tablet: real 0.025s user 0.024s sys 0.000s
I20260924 15:09:08.070564 15747 raft_consensus.cc:359] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.071035 15747 raft_consensus.cc:740] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 43e30527ffcd4c03ad145731b87ec8db, State: Initialized, Role: NON_PARTICIPANT
I20260924 15:09:08.071543 15747 consensus_queue.cc:260] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.073410 15750 tablet_replica.cc:442] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } } }
I20260924 15:09:08.073604 15726 heartbeater.cc:499] Master 127.12.104.254:36199 was elected leader, sending a full tablet report...
I20260924 15:09:08.074043 15750 tablet_replica.cc:445] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260924 15:09:08.074165 15747 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260924 15:09:08.076205 15630 tablet_copy_service.cc:342] P 824a763e0bcf41b6bba4e64c000679af: Request end of tablet copy session 43e30527ffcd4c03ad145731b87ec8db-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:38560
I20260924 15:09:08.076620 15630 tablet_copy_service.cc:434] P 824a763e0bcf41b6bba4e64c000679af: ending tablet copy session 43e30527ffcd4c03ad145731b87ec8db-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer 43e30527ffcd4c03ad145731b87ec8db
W20260924 15:09:08.077134 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260924 15:09:08.082088 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:08.097718 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:08.130987 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:08.132370 12707 tablet_replica.cc:333] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: stopping tablet replica
I20260924 15:09:08.132941 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:08.133483 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:08.147374 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:08.162684 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:08.170375 15751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:08.176223 15752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:08.178742 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:08.179497 15754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:08.180991 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:08.181788 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:08.182044 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262548182021 us; error 0 us; skew 500 ppm
I20260924 15:09:08.182906 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:08.186615 12707 webserver.cc:533] Webserver started at http://127.12.104.193:40493/ using document root <none> and password file <none>
I20260924 15:09:08.187256 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:08.187458 12707 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260924 15:09:08.191983 12707 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 15:09:08.195535 15759 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:08.196663 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:08.197060 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "824a763e0bcf41b6bba4e64c000679af"
format_stamp: "Formatted at 2026-09-24 15:09:07 on dist-test-slave-9164"
I20260924 15:09:08.197481 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:08.227646 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:08.228953 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:08.238530 15768 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20260924 15:09:08.246687 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 15:09:08.247026 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.001s
I20260924 15:09:08.247340 12707 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260924 15:09:08.252851 12707 ts_tablet_manager.cc:616] Registered 1 tablets
I20260924 15:09:08.253149 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.000s sys 0.004s
I20260924 15:09:08.253443 15768 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Bootstrap starting.
I20260924 15:09:08.311767 15768 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 15:09:08.313362 15768 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Bootstrap complete.
I20260924 15:09:08.315470 15768 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Time spent bootstrapping tablet: real 0.062s user 0.047s sys 0.012s
I20260924 15:09:08.320251 15768 raft_consensus.cc:359] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.321030 15768 raft_consensus.cc:740] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824a763e0bcf41b6bba4e64c000679af, State: Initialized, Role: FOLLOWER
I20260924 15:09:08.322295 15768 consensus_queue.cc:260] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.323304 15768 raft_consensus.cc:399] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:08.323711 15768 raft_consensus.cc:493] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:08.324110 15768 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:09:08.335229 15768 raft_consensus.cc:515] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.336304 15768 leader_election.cc:304] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824a763e0bcf41b6bba4e64c000679af; no voters:
I20260924 15:09:08.339635 15768 leader_election.cc:290] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 15:09:08.340054 15806 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 2 FOLLOWER]: Leader election won for term 2
I20260924 15:09:08.350555 15768 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af: Time spent starting tablet: real 0.035s user 0.021s sys 0.011s
I20260924 15:09:08.352123 15806 raft_consensus.cc:697] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 2 LEADER]: Becoming Leader. State: Replica: 824a763e0bcf41b6bba4e64c000679af, State: Running, Role: LEADER
I20260924 15:09:08.353147 15806 consensus_queue.cc:237] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } }
I20260924 15:09:08.357206 15807 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "824a763e0bcf41b6bba4e64c000679af" committed_config { opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } } }
I20260924 15:09:08.357172 15808 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader 824a763e0bcf41b6bba4e64c000679af. Latest consensus state: current_term: 2 leader_uuid: "824a763e0bcf41b6bba4e64c000679af" committed_config { opid_index: -1 peers { permanent_uuid: "824a763e0bcf41b6bba4e64c000679af" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40463 } } }
I20260924 15:09:08.358182 15807 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:08.368690 15814 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:08.369485 15814 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260924 15:09:08.358250 15808 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:08.396694 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:40463
I20260924 15:09:08.396821 15837 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:40463 every 8 connection(s)
I20260924 15:09:08.404889 15838 heartbeater.cc:344] Connected to a master server at 127.12.104.254:36199
I20260924 15:09:08.405411 15838 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:08.407173 15838 heartbeater.cc:507] Master 127.12.104.254:36199 requested a full tablet report, sending...
I20260924 15:09:08.408390 12707 tablet_server.cc:179] TabletServer@127.12.104.193:40463 shutting down...
I20260924 15:09:08.414469 15503 ts_manager.cc:194] Re-registered known tserver with Master: 824a763e0bcf41b6bba4e64c000679af (127.12.104.193:40463)
W20260924 15:09:08.415297 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:08.417120 15503 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39218
I20260924 15:09:08.421481 15838 heartbeater.cc:499] Master 127.12.104.254:36199 was elected leader, sending a full tablet report...
W20260924 15:09:08.427645 15503 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:08.448484 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:08.449525 12707 tablet_replica.cc:333] stopping tablet replica
I20260924 15:09:08.450377 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 2 LEADER]: Raft consensus shutting down.
I20260924 15:09:08.451010 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 824a763e0bcf41b6bba4e64c000679af [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:08.466265 12707 tablet_server.cc:196] TabletServer@127.12.104.193:40463 shutdown complete.
I20260924 15:09:08.479537 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:09:08.511457 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:08.512938 12707 tablet_replica.cc:333] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db: stopping tablet replica
I20260924 15:09:08.513720 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260924 15:09:08.514353 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 43e30527ffcd4c03ad145731b87ec8db [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260924 15:09:08.525341 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:09:08.541448 12707 master.cc:562] Master@127.12.104.254:36199 shutting down...
I20260924 15:09:08.563643 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:08.564296 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:08.564656 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3f419044c09d46cf9c3248e0cbac7414: stopping tablet replica
I20260924 15:09:08.584111 12707 master.cc:584] Master@127.12.104.254:36199 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1327 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260924 15:09:08.612592 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:40885
I20260924 15:09:08.613740 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:08.619644 15844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:08.619917 15845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:08.622681 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:08.624686 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:08.625540 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:08.625759 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:08.625931 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262548625913 us; error 0 us; skew 500 ppm
I20260924 15:09:08.626590 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:08.629622 12707 webserver.cc:533] Webserver started at http://127.12.104.254:45525/ using document root <none> and password file <none>
I20260924 15:09:08.630148 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:08.630389 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:08.630690 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:08.631889 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "94723edaf7904f54abdc44b4c4d8e09e"
format_stamp: "Formatted at 2026-09-24 15:09:08 on dist-test-slave-9164"
I20260924 15:09:08.637019 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260924 15:09:08.641052 15852 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:08.642046 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 15:09:08.642455 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "94723edaf7904f54abdc44b4c4d8e09e"
format_stamp: "Formatted at 2026-09-24 15:09:08 on dist-test-slave-9164"
I20260924 15:09:08.642846 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:08.658653 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:08.660058 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:08.709162 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:40885
I20260924 15:09:08.709255 15903 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:40885 every 8 connection(s)
I20260924 15:09:08.714454 15904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:08.725220 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e: Bootstrap starting.
I20260924 15:09:08.730422 15904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:08.734938 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e: No bootstrap required, opened a new log
I20260924 15:09:08.737092 15904 raft_consensus.cc:359] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER }
I20260924 15:09:08.737494 15904 raft_consensus.cc:385] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:08.737757 15904 raft_consensus.cc:740] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94723edaf7904f54abdc44b4c4d8e09e, State: Initialized, Role: FOLLOWER
I20260924 15:09:08.738453 15904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER }
I20260924 15:09:08.738930 15904 raft_consensus.cc:399] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:08.739156 15904 raft_consensus.cc:493] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:08.739445 15904 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:08.744189 15904 raft_consensus.cc:515] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER }
I20260924 15:09:08.744769 15904 leader_election.cc:304] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94723edaf7904f54abdc44b4c4d8e09e; no voters:
I20260924 15:09:08.746405 15904 leader_election.cc:290] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:08.746778 15907 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:08.748610 15907 raft_consensus.cc:697] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 1 LEADER]: Becoming Leader. State: Replica: 94723edaf7904f54abdc44b4c4d8e09e, State: Running, Role: LEADER
I20260924 15:09:08.749396 15907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER }
I20260924 15:09:08.750133 15904 sys_catalog.cc:565] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:08.752754 15908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94723edaf7904f54abdc44b4c4d8e09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER } }
I20260924 15:09:08.752794 15909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94723edaf7904f54abdc44b4c4d8e09e. Latest consensus state: current_term: 1 leader_uuid: "94723edaf7904f54abdc44b4c4d8e09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94723edaf7904f54abdc44b4c4d8e09e" member_type: VOTER } }
I20260924 15:09:08.753548 15908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:08.753621 15909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:08.771600 15912 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:08.779395 15912 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:08.780607 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:08.790577 15912 catalog_manager.cc:1396] Generated new cluster ID: 7d049213ddc64e449cccdc8021121e0c
I20260924 15:09:08.790930 15912 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:08.806824 15912 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:08.808615 15912 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:08.839754 15912 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e: Generated new TSK 0
I20260924 15:09:08.840468 15912 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:08.847795 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:08.855172 15925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:08.855650 15926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:08.858759 15928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:08.859694 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:08.860563 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:08.860812 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:08.861016 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262548860995 us; error 0 us; skew 500 ppm
I20260924 15:09:08.861611 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:08.864779 12707 webserver.cc:533] Webserver started at http://127.12.104.193:37099/ using document root <none> and password file <none>
I20260924 15:09:08.865360 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:08.865592 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:08.865892 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:08.867141 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "3d0a24dc7ca84b7da08a962c47af497d"
format_stamp: "Formatted at 2026-09-24 15:09:08 on dist-test-slave-9164"
I20260924 15:09:08.872480 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 15:09:08.876554 15933 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:08.877583 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 15:09:08.877913 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "3d0a24dc7ca84b7da08a962c47af497d"
format_stamp: "Formatted at 2026-09-24 15:09:08 on dist-test-slave-9164"
I20260924 15:09:08.878202 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:08.891381 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:08.892560 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:08.905524 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:08.905972 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:08.906433 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:08.906723 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:08.985484 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:43319
I20260924 15:09:08.985595 16004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:43319 every 8 connection(s)
I20260924 15:09:08.990957 16005 heartbeater.cc:344] Connected to a master server at 127.12.104.254:40885
I20260924 15:09:08.991344 16005 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:08.992060 16005 heartbeater.cc:507] Master 127.12.104.254:40885 requested a full tablet report, sending...
I20260924 15:09:08.994114 15869 ts_manager.cc:194] Registered new tserver with Master: 3d0a24dc7ca84b7da08a962c47af497d (127.12.104.193:43319)
I20260924 15:09:08.994231 12707 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.005245903s
I20260924 15:09:08.995899 15869 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32890
I20260924 15:09:09.031652 15969 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260924 15:09:09.033295 15969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:09.050810 16015 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: Bootstrap starting.
I20260924 15:09:09.055876 16015 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:09.060783 16015 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: No bootstrap required, opened a new log
I20260924 15:09:09.061267 16015 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260924 15:09:09.063499 16015 raft_consensus.cc:359] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } }
I20260924 15:09:09.063961 16015 raft_consensus.cc:385] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:09.064244 16015 raft_consensus.cc:740] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0a24dc7ca84b7da08a962c47af497d, State: Initialized, Role: FOLLOWER
I20260924 15:09:09.064914 16015 consensus_queue.cc:260] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } }
I20260924 15:09:09.065475 16015 raft_consensus.cc:399] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:09.065770 16015 raft_consensus.cc:493] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:09.066128 16015 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:09.071555 16015 raft_consensus.cc:515] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } }
I20260924 15:09:09.072170 16015 leader_election.cc:304] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0a24dc7ca84b7da08a962c47af497d; no voters:
I20260924 15:09:09.073643 16015 leader_election.cc:290] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:09.074055 16017 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:09.075763 16005 heartbeater.cc:499] Master 127.12.104.254:40885 was elected leader, sending a full tablet report...
I20260924 15:09:09.076582 16015 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
I20260924 15:09:09.077121 16017 raft_consensus.cc:697] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 1 LEADER]: Becoming Leader. State: Replica: 3d0a24dc7ca84b7da08a962c47af497d, State: Running, Role: LEADER
I20260924 15:09:09.077872 16017 consensus_queue.cc:237] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } }
I20260924 15:09:09.081286 16018 tablet_replica.cc:442] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0a24dc7ca84b7da08a962c47af497d" committed_config { opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } } }
I20260924 15:09:09.081885 16018 tablet_replica.cc:445] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:09.083438 16019 tablet_replica.cc:442] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: TxnStatusTablet state changed. Reason: New leader 3d0a24dc7ca84b7da08a962c47af497d. Latest consensus state: current_term: 1 leader_uuid: "3d0a24dc7ca84b7da08a962c47af497d" committed_config { opid_index: -1 peers { permanent_uuid: "3d0a24dc7ca84b7da08a962c47af497d" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 43319 } } }
I20260924 15:09:09.084085 16019 tablet_replica.cc:445] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: This TxnStatusTablet replica's current role is: LEADER
W20260924 15:09:09.084363 15869 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:09.088613 16021 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:09.089143 16021 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260924 15:09:09.090225 15869 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260924 15:09:09.140852 15956 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260924 15:09:09.141251 15956 txn_status_manager.cc:244] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260924 15:09:09.141659 15956 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260924 15:09:09.150521 15956 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260924 15:09:09.195894 15956 txn_status_manager.cc:244] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260924 15:09:09.196306 15956 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260924 15:09:09.206143 15956 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260924 15:09:09.220118 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.223407 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.226270 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.229632 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.232836 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.235873 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44702
W20260924 15:09:09.248102 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
W20260924 15:09:09.251094 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
W20260924 15:09:09.253983 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
W20260924 15:09:09.256541 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
W20260924 15:09:09.259692 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
W20260924 15:09:09.263028 15969 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44716
I20260924 15:09:09.267485 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:09.304138 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:09.305136 12707 tablet_replica.cc:333] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d: stopping tablet replica
I20260924 15:09:09.305837 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:09.306559 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 3d0a24dc7ca84b7da08a962c47af497d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:09.331830 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:09.347059 12707 master.cc:562] Master@127.12.104.254:40885 shutting down...
I20260924 15:09:09.376801 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:09.377524 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:09.377907 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 94723edaf7904f54abdc44b4c4d8e09e: stopping tablet replica
I20260924 15:09:09.400095 12707 master.cc:584] Master@127.12.104.254:40885 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (818 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260924 15:09:09.431313 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:44065
I20260924 15:09:09.432498 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:09.438422 16030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:09.439221 16031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:09.448084 16033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:09.448333 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:09.450429 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:09.450663 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:09.450896 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262549450854 us; error 0 us; skew 500 ppm
I20260924 15:09:09.451547 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:09.454665 12707 webserver.cc:533] Webserver started at http://127.12.104.254:42529/ using document root <none> and password file <none>
I20260924 15:09:09.455224 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:09.455454 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:09.455775 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:09.457149 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "970da864a8f54ee8b8cb45cdc22abfe2"
format_stamp: "Formatted at 2026-09-24 15:09:09 on dist-test-slave-9164"
I20260924 15:09:09.462988 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260924 15:09:09.467212 16038 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:09.468158 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 15:09:09.468521 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "970da864a8f54ee8b8cb45cdc22abfe2"
format_stamp: "Formatted at 2026-09-24 15:09:09 on dist-test-slave-9164"
I20260924 15:09:09.468855 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:09.520921 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:09.522004 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:09.567960 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:44065
I20260924 15:09:09.568073 16096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:44065 every 8 connection(s)
I20260924 15:09:09.572237 16097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:09.584123 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2: Bootstrap starting.
I20260924 15:09:09.588956 16097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:09.593242 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2: No bootstrap required, opened a new log
I20260924 15:09:09.595443 16097 raft_consensus.cc:359] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER }
I20260924 15:09:09.595849 16097 raft_consensus.cc:385] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:09.596159 16097 raft_consensus.cc:740] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970da864a8f54ee8b8cb45cdc22abfe2, State: Initialized, Role: FOLLOWER
I20260924 15:09:09.596848 16097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER }
I20260924 15:09:09.597329 16097 raft_consensus.cc:399] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:09.597563 16097 raft_consensus.cc:493] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:09.597869 16097 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:09.603024 16097 raft_consensus.cc:515] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER }
I20260924 15:09:09.603614 16097 leader_election.cc:304] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 970da864a8f54ee8b8cb45cdc22abfe2; no voters:
I20260924 15:09:09.604941 16097 leader_election.cc:290] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:09.605350 16100 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:09.606789 16100 raft_consensus.cc:697] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 1 LEADER]: Becoming Leader. State: Replica: 970da864a8f54ee8b8cb45cdc22abfe2, State: Running, Role: LEADER
I20260924 15:09:09.607522 16100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER }
I20260924 15:09:09.608150 16097 sys_catalog.cc:565] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:09.611922 16102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 970da864a8f54ee8b8cb45cdc22abfe2. Latest consensus state: current_term: 1 leader_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER } }
I20260924 15:09:09.612639 16102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:09.613816 16101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970da864a8f54ee8b8cb45cdc22abfe2" member_type: VOTER } }
I20260924 15:09:09.614393 16101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:09.614566 16107 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:09.619459 16107 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:09.625309 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:09.628925 16107 catalog_manager.cc:1396] Generated new cluster ID: 8fda91247f9641e0a9c66f30c6f9e3d8
I20260924 15:09:09.629241 16107 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:09.659237 16107 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:09.660529 16107 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:09.680416 16107 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2: Generated new TSK 0
I20260924 15:09:09.681052 16107 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:09.692498 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:09.698782 16118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:09.700050 16119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:09.702971 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:09.702893 16121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:09.704202 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:09.704445 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:09.704638 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262549704615 us; error 0 us; skew 500 ppm
I20260924 15:09:09.705178 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:09.707700 12707 webserver.cc:533] Webserver started at http://127.12.104.193:33773/ using document root <none> and password file <none>
I20260924 15:09:09.708230 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:09.708448 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:09.708755 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:09.710227 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "22a73a161cf54593a5bac07918d76805"
format_stamp: "Formatted at 2026-09-24 15:09:09 on dist-test-slave-9164"
I20260924 15:09:09.715232 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260924 15:09:09.718981 16126 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:09.719909 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 15:09:09.720271 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "22a73a161cf54593a5bac07918d76805"
format_stamp: "Formatted at 2026-09-24 15:09:09 on dist-test-slave-9164"
I20260924 15:09:09.720640 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:09.740645 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:09.742358 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:09.753758 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:09.754134 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:09.754506 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:09.754742 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:09.808911 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:40559
I20260924 15:09:09.809051 16196 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:40559 every 8 connection(s)
I20260924 15:09:09.814275 16197 heartbeater.cc:344] Connected to a master server at 127.12.104.254:44065
I20260924 15:09:09.814751 16197 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:09.815547 16197 heartbeater.cc:507] Master 127.12.104.254:44065 requested a full tablet report, sending...
I20260924 15:09:09.817653 16055 ts_manager.cc:194] Registered new tserver with Master: 22a73a161cf54593a5bac07918d76805 (127.12.104.193:40559)
I20260924 15:09:09.818651 12707 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006126051s
I20260924 15:09:09.819603 16055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50158
I20260924 15:09:09.850845 16162 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260924 15:09:09.852063 16162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:09.864586 16208 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: Bootstrap starting.
I20260924 15:09:09.869371 16208 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:09.873965 16208 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: No bootstrap required, opened a new log
I20260924 15:09:09.874501 16208 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260924 15:09:09.876623 16208 raft_consensus.cc:359] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.877063 16208 raft_consensus.cc:385] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:09.877362 16208 raft_consensus.cc:740] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a73a161cf54593a5bac07918d76805, State: Initialized, Role: FOLLOWER
I20260924 15:09:09.878021 16208 consensus_queue.cc:260] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.878599 16208 raft_consensus.cc:399] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:09.878839 16208 raft_consensus.cc:493] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:09.879118 16208 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:09.884749 16208 raft_consensus.cc:515] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.885335 16208 leader_election.cc:304] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22a73a161cf54593a5bac07918d76805; no voters:
I20260924 15:09:09.886647 16208 leader_election.cc:290] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:09.887046 16210 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:09.888413 16210 raft_consensus.cc:697] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 1 LEADER]: Becoming Leader. State: Replica: 22a73a161cf54593a5bac07918d76805, State: Running, Role: LEADER
I20260924 15:09:09.888556 16197 heartbeater.cc:499] Master 127.12.104.254:44065 was elected leader, sending a full tablet report...
I20260924 15:09:09.889107 16208 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260924 15:09:09.889197 16210 consensus_queue.cc:237] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.893105 16212 tablet_replica.cc:442] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: TxnStatusTablet state changed. Reason: New leader 22a73a161cf54593a5bac07918d76805. Latest consensus state: current_term: 1 leader_uuid: "22a73a161cf54593a5bac07918d76805" committed_config { opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } } }
I20260924 15:09:09.893774 16212 tablet_replica.cc:445] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: This TxnStatusTablet replica's current role is: LEADER
I20260924 15:09:09.895463 16211 tablet_replica.cc:442] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22a73a161cf54593a5bac07918d76805" committed_config { opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } } }
I20260924 15:09:09.895946 16211 tablet_replica.cc:445] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: This TxnStatusTablet replica's current role is: LEADER
W20260924 15:09:09.897929 16055 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:09.902729 16215 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260924 15:09:09.903196 16215 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260924 15:09:09.903959 16055 catalog_manager.cc:5671] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260924 15:09:09.928611 16162 tablet_service.cc:1511] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260924 15:09:09.929705 16162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:09.940387 16208 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: Bootstrap starting.
I20260924 15:09:09.945168 16208 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:09.949551 16208 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: No bootstrap required, opened a new log
I20260924 15:09:09.950018 16208 ts_tablet_manager.cc:1403] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260924 15:09:09.952170 16208 raft_consensus.cc:359] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.952625 16208 raft_consensus.cc:385] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:09.952867 16208 raft_consensus.cc:740] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a73a161cf54593a5bac07918d76805, State: Initialized, Role: FOLLOWER
I20260924 15:09:09.953435 16208 consensus_queue.cc:260] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.953909 16208 raft_consensus.cc:399] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:09.954164 16208 raft_consensus.cc:493] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:09.954533 16208 raft_consensus.cc:3060] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:09.959357 16208 raft_consensus.cc:515] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.959975 16208 leader_election.cc:304] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22a73a161cf54593a5bac07918d76805; no voters:
I20260924 15:09:09.960526 16208 leader_election.cc:290] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:09.960723 16210 raft_consensus.cc:2804] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:09.961299 16210 raft_consensus.cc:697] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 1 LEADER]: Becoming Leader. State: Replica: 22a73a161cf54593a5bac07918d76805, State: Running, Role: LEADER
I20260924 15:09:09.962010 16210 consensus_queue.cc:237] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "22a73a161cf54593a5bac07918d76805" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 40559 } }
I20260924 15:09:09.963035 16208 ts_tablet_manager.cc:1434] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
W20260924 15:09:09.967568 16055 catalog_manager.cc:5671] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260924 15:09:09.971715 16055 catalog_manager.cc:5671] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260924 15:09:09.992590 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:10.015103 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:10.015987 12707 tablet_replica.cc:333] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805: stopping tablet replica
I20260924 15:09:10.016510 12707 raft_consensus.cc:2243] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:10.016911 12707 raft_consensus.cc:2272] T 22222222222222222222222222222222 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:10.019646 12707 tablet_replica.cc:333] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805: stopping tablet replica
I20260924 15:09:10.020121 12707 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:10.020530 12707 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 22a73a161cf54593a5bac07918d76805 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:10.032701 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:10.044909 12707 master.cc:562] Master@127.12.104.254:44065 shutting down...
I20260924 15:09:10.061342 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:10.061928 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:10.062463 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 970da864a8f54ee8b8cb45cdc22abfe2: stopping tablet replica
I20260924 15:09:10.082098 12707 master.cc:584] Master@127.12.104.254:44065 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (677 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3726 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260924 15:09:10.108317 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:38479
I20260924 15:09:10.109423 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:10.114434 16224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.115654 16225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.117731 16227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:10.117868 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:10.118772 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:10.119026 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:10.119200 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262550119184 us; error 0 us; skew 500 ppm
I20260924 15:09:10.119740 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:10.122036 12707 webserver.cc:533] Webserver started at http://127.12.104.254:38435/ using document root <none> and password file <none>
I20260924 15:09:10.122671 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:10.122889 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:10.123174 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:10.124291 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.128867 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 15:09:10.132340 16232 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.133097 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 15:09:10.133400 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.133706 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:10.153441 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:10.154603 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:10.191417 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:38479
I20260924 15:09:10.191547 16283 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:38479 every 8 connection(s)
I20260924 15:09:10.195461 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:10.206151 16284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4: Bootstrap starting.
I20260924 15:09:10.210669 16284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:10.214723 16284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4: No bootstrap required, opened a new log
I20260924 15:09:10.216799 16284 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER }
I20260924 15:09:10.217224 16284 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:10.217486 16284 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cc8a8e5ca4e22b69c2e5605998ed4, State: Initialized, Role: FOLLOWER
I20260924 15:09:10.218078 16284 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER }
I20260924 15:09:10.218583 16284 raft_consensus.cc:399] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:10.218819 16284 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:10.219105 16284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:10.224259 16284 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER }
I20260924 15:09:10.224901 16284 leader_election.cc:304] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc8cc8a8e5ca4e22b69c2e5605998ed4; no voters:
I20260924 15:09:10.226193 16284 leader_election.cc:290] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:10.226652 16287 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:10.228024 16287 raft_consensus.cc:697] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 1 LEADER]: Becoming Leader. State: Replica: fc8cc8a8e5ca4e22b69c2e5605998ed4, State: Running, Role: LEADER
I20260924 15:09:10.228808 16287 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER }
I20260924 15:09:10.229328 16284 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:10.231619 16288 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER } }
I20260924 15:09:10.231619 16289 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc8cc8a8e5ca4e22b69c2e5605998ed4. Latest consensus state: current_term: 1 leader_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc8cc8a8e5ca4e22b69c2e5605998ed4" member_type: VOTER } }
I20260924 15:09:10.232437 16288 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:10.232546 16289 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:10.236200 16292 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:10.241359 16292 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:10.244954 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:10.251891 16292 catalog_manager.cc:1396] Generated new cluster ID: 31fa0e5f31f048eda73cd5687c84d0ac
I20260924 15:09:10.252272 16292 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:10.279297 16292 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:10.280730 16292 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:10.302011 16292 catalog_manager.cc:6219] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4: Generated new TSK 0
I20260924 15:09:10.302975 16292 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:10.314564 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:10.320971 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.322950 16306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.325599 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:10.326431 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:10.327277 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:10.327481 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:10.327618 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262550327606 us; error 0 us; skew 500 ppm
I20260924 15:09:10.328146 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:10.331552 12707 webserver.cc:533] Webserver started at http://127.12.104.193:33521/ using document root <none> and password file <none>
I20260924 15:09:10.332103 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:10.332284 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:10.332523 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:10.333711 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "c721bd643fa948efa627bf307e05b28c"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.339404 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 15:09:10.343276 16313 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.344286 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 15:09:10.344642 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "c721bd643fa948efa627bf307e05b28c"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.344961 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:10.372900 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:10.374344 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:10.376148 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:10.379329 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:10.379544 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.379832 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:10.380051 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.429417 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:35911
I20260924 15:09:10.429533 16376 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:35911 every 8 connection(s)
I20260924 15:09:10.435117 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:10.443231 16381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.445364 16382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.446103 16384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:10.448094 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:10.449182 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:10.449421 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:10.449615 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262550449580 us; error 0 us; skew 500 ppm
I20260924 15:09:10.450186 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:10.452719 16377 heartbeater.cc:344] Connected to a master server at 127.12.104.254:38479
I20260924 15:09:10.453131 16377 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:10.453970 16377 heartbeater.cc:507] Master 127.12.104.254:38479 requested a full tablet report, sending...
I20260924 15:09:10.454355 12707 webserver.cc:533] Webserver started at http://127.12.104.194:45599/ using document root <none> and password file <none>
I20260924 15:09:10.455000 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:10.455217 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:10.455560 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:10.456418 16249 ts_manager.cc:194] Registered new tserver with Master: c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911)
I20260924 15:09:10.456907 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "953538a96d5f4e0586e2d8fafe939193"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.458503 16249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56338
I20260924 15:09:10.462451 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260924 15:09:10.467145 16389 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.468299 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 15:09:10.468626 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "953538a96d5f4e0586e2d8fafe939193"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.468912 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:10.498843 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:10.499986 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:10.501572 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:10.504132 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:10.504341 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.504668 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:10.504869 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.546998 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:40755
I20260924 15:09:10.547107 16451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:40755 every 8 connection(s)
I20260924 15:09:10.552238 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:10.561205 16455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.562712 16456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:10.565529 16458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:10.566156 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:10.566933 16452 heartbeater.cc:344] Connected to a master server at 127.12.104.254:38479
I20260924 15:09:10.567312 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260924 15:09:10.567325 16452 heartbeater.cc:461] Registering TS with master...
W20260924 15:09:10.567655 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:10.567921 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262550567889 us; error 0 us; skew 500 ppm
I20260924 15:09:10.568395 16452 heartbeater.cc:507] Master 127.12.104.254:38479 requested a full tablet report, sending...
I20260924 15:09:10.568545 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:10.570765 16249 ts_manager.cc:194] Registered new tserver with Master: 953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755)
I20260924 15:09:10.571621 12707 webserver.cc:533] Webserver started at http://127.12.104.195:45289/ using document root <none> and password file <none>
I20260924 15:09:10.572161 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:10.572364 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:10.572438 16249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56346
I20260924 15:09:10.572660 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:10.574111 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-2-root/instance:
uuid: "9035a5477f7c4ed583fb09f0982aef23"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.578713 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 15:09:10.582087 16463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.582973 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260924 15:09:10.583292 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-2-root
uuid: "9035a5477f7c4ed583fb09f0982aef23"
format_stamp: "Formatted at 2026-09-24 15:09:10 on dist-test-slave-9164"
I20260924 15:09:10.583619 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:10.595008 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:10.596156 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:10.597627 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:10.600082 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:10.600330 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.600589 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:10.600766 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:10.640381 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.195:39623
I20260924 15:09:10.640499 16525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.195:39623 every 8 connection(s)
I20260924 15:09:10.654603 16526 heartbeater.cc:344] Connected to a master server at 127.12.104.254:38479
I20260924 15:09:10.655037 16526 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:10.655824 16526 heartbeater.cc:507] Master 127.12.104.254:38479 requested a full tablet report, sending...
I20260924 15:09:10.658237 16249 ts_manager.cc:194] Registered new tserver with Master: 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623)
I20260924 15:09:10.658790 12707 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014993623s
I20260924 15:09:10.660282 16249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56348
I20260924 15:09:10.706475 16249 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56374:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 15:09:10.758810 16417 tablet_service.cc:1511] Processing CreateTablet for tablet 2d757ecbc6d245d19d1cd8575d3d0651 (DEFAULT_TABLE table=test-workload [id=1826597ebd5c426dbd0fd7c6aebbd47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:10.760210 16417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d757ecbc6d245d19d1cd8575d3d0651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:10.760331 16342 tablet_service.cc:1511] Processing CreateTablet for tablet 2d757ecbc6d245d19d1cd8575d3d0651 (DEFAULT_TABLE table=test-workload [id=1826597ebd5c426dbd0fd7c6aebbd47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:10.761863 16342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d757ecbc6d245d19d1cd8575d3d0651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:10.769104 16491 tablet_service.cc:1511] Processing CreateTablet for tablet 2d757ecbc6d245d19d1cd8575d3d0651 (DEFAULT_TABLE table=test-workload [id=1826597ebd5c426dbd0fd7c6aebbd47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:10.770666 16491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d757ecbc6d245d19d1cd8575d3d0651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:10.777704 16546 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: Bootstrap starting.
I20260924 15:09:10.780557 16547 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Bootstrap starting.
I20260924 15:09:10.785382 16546 tablet_bootstrap.cc:654] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:10.788271 16547 tablet_bootstrap.cc:654] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:10.788411 16548 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Bootstrap starting.
I20260924 15:09:10.792133 16546 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: No bootstrap required, opened a new log
I20260924 15:09:10.792698 16546 ts_tablet_manager.cc:1403] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260924 15:09:10.794279 16547 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: No bootstrap required, opened a new log
I20260924 15:09:10.794849 16547 ts_tablet_manager.cc:1403] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260924 15:09:10.794968 16548 tablet_bootstrap.cc:654] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:10.795929 16546 raft_consensus.cc:359] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.796725 16546 raft_consensus.cc:385] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:10.797041 16546 raft_consensus.cc:740] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953538a96d5f4e0586e2d8fafe939193, State: Initialized, Role: FOLLOWER
I20260924 15:09:10.797873 16547 raft_consensus.cc:359] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.797978 16546 consensus_queue.cc:260] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.798784 16547 raft_consensus.cc:385] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:10.799115 16547 raft_consensus.cc:740] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c721bd643fa948efa627bf307e05b28c, State: Initialized, Role: FOLLOWER
I20260924 15:09:10.800259 16547 consensus_queue.cc:260] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.804298 16452 heartbeater.cc:499] Master 127.12.104.254:38479 was elected leader, sending a full tablet report...
I20260924 15:09:10.804935 16548 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: No bootstrap required, opened a new log
I20260924 15:09:10.804304 16547 ts_tablet_manager.cc:1434] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Time spent starting tablet: real 0.009s user 0.002s sys 0.007s
I20260924 15:09:10.805745 16548 ts_tablet_manager.cc:1403] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20260924 15:09:10.806246 16377 heartbeater.cc:499] Master 127.12.104.254:38479 was elected leader, sending a full tablet report...
I20260924 15:09:10.808758 16548 raft_consensus.cc:359] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.809378 16548 raft_consensus.cc:385] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:10.809653 16548 raft_consensus.cc:740] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9035a5477f7c4ed583fb09f0982aef23, State: Initialized, Role: FOLLOWER
I20260924 15:09:10.810401 16548 consensus_queue.cc:260] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.804888 16546 ts_tablet_manager.cc:1434] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260924 15:09:10.813271 16526 heartbeater.cc:499] Master 127.12.104.254:38479 was elected leader, sending a full tablet report...
I20260924 15:09:10.813750 16548 ts_tablet_manager.cc:1434] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260924 15:09:10.826251 16553 raft_consensus.cc:493] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:09:10.826817 16553 raft_consensus.cc:515] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.829028 16553 leader_election.cc:290] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755), 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623)
I20260924 15:09:10.838470 16427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "c721bd643fa948efa627bf307e05b28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953538a96d5f4e0586e2d8fafe939193" is_pre_election: true
I20260924 15:09:10.839218 16427 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c721bd643fa948efa627bf307e05b28c in term 0.
I20260924 15:09:10.840266 16501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "c721bd643fa948efa627bf307e05b28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9035a5477f7c4ed583fb09f0982aef23" is_pre_election: true
I20260924 15:09:10.840390 16314 leader_election.cc:304] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953538a96d5f4e0586e2d8fafe939193, c721bd643fa948efa627bf307e05b28c; no voters:
I20260924 15:09:10.840912 16501 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c721bd643fa948efa627bf307e05b28c in term 0.
I20260924 15:09:10.841214 16553 raft_consensus.cc:2804] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:09:10.841599 16553 raft_consensus.cc:493] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:09:10.841956 16553 raft_consensus.cc:3060] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:10.847306 16553 raft_consensus.cc:515] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.848932 16553 leader_election.cc:290] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [CANDIDATE]: Term 1 election: Requested vote from peers 953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755), 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623)
I20260924 15:09:10.849931 16427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "c721bd643fa948efa627bf307e05b28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953538a96d5f4e0586e2d8fafe939193"
I20260924 15:09:10.849968 16501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "c721bd643fa948efa627bf307e05b28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9035a5477f7c4ed583fb09f0982aef23"
I20260924 15:09:10.850641 16427 raft_consensus.cc:3060] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:10.850754 16501 raft_consensus.cc:3060] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:10.856771 16501 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c721bd643fa948efa627bf307e05b28c in term 1.
I20260924 15:09:10.856760 16427 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c721bd643fa948efa627bf307e05b28c in term 1.
I20260924 15:09:10.858130 16316 leader_election.cc:304] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9035a5477f7c4ed583fb09f0982aef23, c721bd643fa948efa627bf307e05b28c; no voters:
I20260924 15:09:10.858982 16553 raft_consensus.cc:2804] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:10.859982 16553 raft_consensus.cc:697] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 1 LEADER]: Becoming Leader. State: Replica: c721bd643fa948efa627bf307e05b28c, State: Running, Role: LEADER
I20260924 15:09:10.860694 16553 consensus_queue.cc:237] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:10.869583 16248 catalog_manager.cc:5838] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c reported cstate change: term changed from 0 to 1, leader changed from <none> to c721bd643fa948efa627bf307e05b28c (127.12.104.193). New cstate: current_term: 1 leader_uuid: "c721bd643fa948efa627bf307e05b28c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } }
I20260924 15:09:11.131729 16491 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:11.132918 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:11.134733 16342 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:11.353893 16553 consensus_queue.cc:1048] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:09:11.370683 16586 consensus_queue.cc:1048] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 0 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 2d757ecbc6d245d19d1cd8575d3d0651 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755): RUNNING
c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): RUNNING [LEADER]
9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623): RUNNING
All reported replicas are:
A = 953538a96d5f4e0586e2d8fafe939193
B = c721bd643fa948efa627bf307e05b28c
C = 9035a5477f7c4ed583fb09f0982aef23
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:11.399226 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:11.789091 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:11.789361 16491 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:11.793756 16342 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 0 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260924 15:09:12.016254 12707 log_verifier.cc:126] Checking tablet 2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:12.081044 12707 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:12.081614 12707 tablet_replica.cc:333] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: stopping tablet replica
I20260924 15:09:12.082085 12707 raft_consensus.cc:2243] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:12.083029 12707 raft_consensus.cc:2272] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:12.311484 16491 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:12.311800 16342 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:12.312583 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 0 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 2d757ecbc6d245d19d1cd8575d3d0651 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755): RUNNING
c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623): RUNNING
All reported replicas are:
A = 953538a96d5f4e0586e2d8fafe939193
B = c721bd643fa948efa627bf307e05b28c
C = 9035a5477f7c4ed583fb09f0982aef23
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:12.538002 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:12.944988 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:12.945314 16491 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:12.946468 16342 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 0 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 2d757ecbc6d245d19d1cd8575d3d0651 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755): RUNNING
c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623): RUNNING
All reported replicas are:
A = 953538a96d5f4e0586e2d8fafe939193
B = c721bd643fa948efa627bf307e05b28c
C = 9035a5477f7c4ed583fb09f0982aef23
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:13.189363 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:13.458696 16673 raft_consensus.cc:493] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c721bd643fa948efa627bf307e05b28c)
I20260924 15:09:13.459983 16674 raft_consensus.cc:493] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c721bd643fa948efa627bf307e05b28c)
I20260924 15:09:13.460474 16673 raft_consensus.cc:515] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:13.461038 16674 raft_consensus.cc:515] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:13.464670 16673 leader_election.cc:290] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755), c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911)
I20260924 15:09:13.473888 16674 leader_election.cc:290] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911), 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623)
I20260924 15:09:13.507369 16352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "9035a5477f7c4ed583fb09f0982aef23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c721bd643fa948efa627bf307e05b28c" is_pre_election: true
W20260924 15:09:13.511212 16466 leader_election.cc:343] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): Illegal state: cannot vote while shut down
I20260924 15:09:13.513321 16352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "953538a96d5f4e0586e2d8fafe939193" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c721bd643fa948efa627bf307e05b28c" is_pre_election: true
W20260924 15:09:13.515265 16392 leader_election.cc:343] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): Illegal state: cannot vote while shut down
I20260924 15:09:13.517750 16427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "9035a5477f7c4ed583fb09f0982aef23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "953538a96d5f4e0586e2d8fafe939193" is_pre_election: true
I20260924 15:09:13.518708 16427 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9035a5477f7c4ed583fb09f0982aef23 in term 1.
I20260924 15:09:13.521574 16464 leader_election.cc:304] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9035a5477f7c4ed583fb09f0982aef23, 953538a96d5f4e0586e2d8fafe939193; no voters: c721bd643fa948efa627bf307e05b28c
I20260924 15:09:13.523263 16673 raft_consensus.cc:2804] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 15:09:13.523111 16501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "953538a96d5f4e0586e2d8fafe939193" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9035a5477f7c4ed583fb09f0982aef23" is_pre_election: true
I20260924 15:09:13.523813 16673 raft_consensus.cc:493] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c721bd643fa948efa627bf307e05b28c)
I20260924 15:09:13.524341 16673 raft_consensus.cc:3060] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:09:13.535429 16673 raft_consensus.cc:515] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:13.537128 16501 raft_consensus.cc:2393] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 953538a96d5f4e0586e2d8fafe939193 in current term 2: Already voted for candidate 9035a5477f7c4ed583fb09f0982aef23 in this term.
I20260924 15:09:13.539278 16673 leader_election.cc:290] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 election: Requested vote from peers 953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755), c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911)
I20260924 15:09:13.540460 16352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "9035a5477f7c4ed583fb09f0982aef23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c721bd643fa948efa627bf307e05b28c"
W20260924 15:09:13.542887 16466 leader_election.cc:343] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): Illegal state: cannot vote while shut down
I20260924 15:09:13.542666 16427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d757ecbc6d245d19d1cd8575d3d0651" candidate_uuid: "9035a5477f7c4ed583fb09f0982aef23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "953538a96d5f4e0586e2d8fafe939193"
I20260924 15:09:13.543877 16427 raft_consensus.cc:3060] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 1 FOLLOWER]: Advancing to term 2
I20260924 15:09:13.545629 16392 leader_election.cc:304] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 953538a96d5f4e0586e2d8fafe939193; no voters: 9035a5477f7c4ed583fb09f0982aef23, c721bd643fa948efa627bf307e05b28c
I20260924 15:09:13.551745 16427 raft_consensus.cc:2468] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9035a5477f7c4ed583fb09f0982aef23 in term 2.
I20260924 15:09:13.552345 16674 raft_consensus.cc:2749] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 15:09:13.553048 16464 leader_election.cc:304] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9035a5477f7c4ed583fb09f0982aef23, 953538a96d5f4e0586e2d8fafe939193; no voters: c721bd643fa948efa627bf307e05b28c
I20260924 15:09:13.554129 16673 raft_consensus.cc:2804] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 15:09:13.556945 16673 raft_consensus.cc:697] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 LEADER]: Becoming Leader. State: Replica: 9035a5477f7c4ed583fb09f0982aef23, State: Running, Role: LEADER
I20260924 15:09:13.558846 16673 consensus_queue.cc:237] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:13.573249 16248 catalog_manager.cc:5838] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 reported cstate change: term changed from 1 to 2, leader changed from c721bd643fa948efa627bf307e05b28c (127.12.104.193) to 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195). New cstate: current_term: 2 leader_uuid: "9035a5477f7c4ed583fb09f0982aef23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } }
I20260924 15:09:13.811439 16491 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:13.812668 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:13.814810 16342 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20260924 15:09:13.994442 16466 consensus_peers.cc:597] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 -> Peer c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): Couldn't send request to peer c721bd643fa948efa627bf307e05b28c. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:14.044096 16427 raft_consensus.cc:1275] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Refusing update from remote peer 9035a5477f7c4ed583fb09f0982aef23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 15:09:14.045840 16673 consensus_queue.cc:1048] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 1 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 2d757ecbc6d245d19d1cd8575d3d0651 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755): RUNNING
c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623): RUNNING [LEADER]
All reported replicas are:
A = 953538a96d5f4e0586e2d8fafe939193
B = c721bd643fa948efa627bf307e05b28c
C = 9035a5477f7c4ed583fb09f0982aef23
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 2 | -1 | Yes
B | [config not available] | | |
C | A B C* | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:14.064699 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:14.519038 16714 raft_consensus.cc:1012] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Attempting to remove follower c721bd643fa948efa627bf307e05b28c from the Raft config. Reason: The tablet replica hosted on peer c721bd643fa948efa627bf307e05b28c has failed
I20260924 15:09:14.521353 16714 consensus_queue.cc:237] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } }
I20260924 15:09:14.523801 16673 raft_consensus.cc:966] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 LEADER]: Processing failure of peer c721bd643fa948efa627bf307e05b28c in term 2 (The tablet replica hosted on peer c721bd643fa948efa627bf307e05b28c has failed): There is already a config change operation in progress. Unable to evict follower until it completes. Doing nothing.
I20260924 15:09:14.526130 16427 raft_consensus.cc:1275] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Refusing update from remote peer 9035a5477f7c4ed583fb09f0982aef23: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 15:09:14.527650 16673 consensus_queue.cc:1048] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 15:09:14.533986 16724 raft_consensus.cc:2955] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } }
I20260924 15:09:14.535454 16427 raft_consensus.cc:2955] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } }
I20260924 15:09:14.546995 16247 catalog_manager.cc:5838] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 reported cstate change: config changed from index -1 to 3, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) evicted. New cstate: current_term: 2 leader_uuid: "9035a5477f7c4ed583fb09f0982aef23" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } }
I20260924 15:09:14.560381 16342 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d757ecbc6d245d19d1cd8575d3d0651 with delete_type TABLET_DATA_TOMBSTONED (TS c721bd643fa948efa627bf307e05b28c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:41960
I20260924 15:09:14.563937 16728 ts_tablet_manager.cc:1916] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 15:09:14.564399 16501 consensus_queue.cc:237] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } }
W20260924 15:09:14.572247 16466 consensus_peers.cc:597] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 -> Peer c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): Couldn't send request to peer c721bd643fa948efa627bf307e05b28c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:14.574461 16427 raft_consensus.cc:1275] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Refusing update from remote peer 9035a5477f7c4ed583fb09f0982aef23: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 15:09:14.576479 16724 consensus_queue.cc:1048] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 15:09:14.583926 16728 ts_tablet_manager.cc:1929] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260924 15:09:14.584394 16728 log.cc:1199] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790262492846473-12707-0/minicluster-data/ts-0-root/wals/2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:14.584916 16724 raft_consensus.cc:2955] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } }
I20260924 15:09:14.586501 16235 catalog_manager.cc:5063] TS c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): tablet 2d757ecbc6d245d19d1cd8575d3d0651 (table test-workload [id=1826597ebd5c426dbd0fd7c6aebbd47b]) successfully deleted
I20260924 15:09:14.587468 16427 raft_consensus.cc:2955] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } }
I20260924 15:09:14.595602 16235 catalog_manager.cc:5245] ChangeConfig:ADD_PEER:VOTER RPC for tablet 2d757ecbc6d245d19d1cd8575d3d0651 with cas_config_opid_index 3: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260924 15:09:14.600871 16247 catalog_manager.cc:5838] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 reported cstate change: config changed from index 3 to 4, VOTER c721bd643fa948efa627bf307e05b28c (127.12.104.193) added. New cstate: current_term: 2 leader_uuid: "9035a5477f7c4ed583fb09f0982aef23" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } } }
I20260924 15:09:14.679971 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:14.681473 16491 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:14.686860 16342 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 1 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 2d757ecbc6d245d19d1cd8575d3d0651 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
953538a96d5f4e0586e2d8fafe939193 (127.12.104.194:40755): RUNNING
9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623): RUNNING [LEADER]
c721bd643fa948efa627bf307e05b28c (127.12.104.193:35911): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = 953538a96d5f4e0586e2d8fafe939193
B = 9035a5477f7c4ed583fb09f0982aef23
C = c721bd643fa948efa627bf307e05b28c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 4 | Yes
B | A B* C | 2 | 4 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:14.918699 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:15.188526 16749 ts_tablet_manager.cc:933] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Initiating tablet copy from peer 9035a5477f7c4ed583fb09f0982aef23 (127.12.104.195:39623)
I20260924 15:09:15.189919 16749 tablet_copy_client.cc:323] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet copy: Beginning tablet copy session from remote peer at address 127.12.104.195:39623
I20260924 15:09:15.191685 16511 tablet_copy_service.cc:140] P 9035a5477f7c4ed583fb09f0982aef23: Received BeginTabletCopySession request for tablet 2d757ecbc6d245d19d1cd8575d3d0651 from peer c721bd643fa948efa627bf307e05b28c ({username='slave'} at 127.0.0.1:48884)
I20260924 15:09:15.192173 16511 tablet_copy_service.cc:161] P 9035a5477f7c4ed583fb09f0982aef23: Beginning new tablet copy session on tablet 2d757ecbc6d245d19d1cd8575d3d0651 from peer c721bd643fa948efa627bf307e05b28c at {username='slave'} at 127.0.0.1:48884: session id = c721bd643fa948efa627bf307e05b28c-2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:15.197840 16511 tablet_copy_source_session.cc:215] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 15:09:15.200395 16749 ts_tablet_manager.cc:1916] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260924 15:09:15.211855 16749 ts_tablet_manager.cc:1929] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260924 15:09:15.212508 16749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d757ecbc6d245d19d1cd8575d3d0651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:15.218039 16749 tablet_copy_client.cc:806] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet copy: Starting download of 0 data blocks...
I20260924 15:09:15.218583 16749 tablet_copy_client.cc:670] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet copy: Starting download of 1 WAL segments...
I20260924 15:09:15.222134 16749 tablet_copy_client.cc:538] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 15:09:15.228688 16749 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Bootstrap starting.
I20260924 15:09:15.247335 16749 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 15:09:15.248183 16749 tablet_bootstrap.cc:492] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Bootstrap complete.
I20260924 15:09:15.248814 16749 ts_tablet_manager.cc:1403] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.001s
I20260924 15:09:15.251024 16749 raft_consensus.cc:359] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } }
I20260924 15:09:15.251607 16749 raft_consensus.cc:740] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c721bd643fa948efa627bf307e05b28c, State: Initialized, Role: FOLLOWER
I20260924 15:09:15.252252 16749 consensus_queue.cc:260] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953538a96d5f4e0586e2d8fafe939193" member_type: VOTER last_known_addr { host: "127.12.104.194" port: 40755 } } peers { permanent_uuid: "9035a5477f7c4ed583fb09f0982aef23" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39623 } } peers { permanent_uuid: "c721bd643fa948efa627bf307e05b28c" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35911 } }
I20260924 15:09:15.254550 16749 ts_tablet_manager.cc:1434] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260924 15:09:15.256628 16511 tablet_copy_service.cc:342] P 9035a5477f7c4ed583fb09f0982aef23: Request end of tablet copy session c721bd643fa948efa627bf307e05b28c-2d757ecbc6d245d19d1cd8575d3d0651 received from {username='slave'} at 127.0.0.1:48884
I20260924 15:09:15.257011 16511 tablet_copy_service.cc:434] P 9035a5477f7c4ed583fb09f0982aef23: ending tablet copy session c721bd643fa948efa627bf307e05b28c-2d757ecbc6d245d19d1cd8575d3d0651 on tablet 2d757ecbc6d245d19d1cd8575d3d0651 with peer c721bd643fa948efa627bf307e05b28c
I20260924 15:09:15.672389 16491 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:15.681913 16352 raft_consensus.cc:1217] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.3->[2.4-2.4] Dedup: 2.4->[]
I20260924 15:09:15.686074 16417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:15.694857 16342 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
fc8cc8a8e5ca4e22b69c2e5605998ed4 | 127.12.104.254:38479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
9035a5477f7c4ed583fb09f0982aef23 | 127.12.104.195:39623 | HEALTHY | <none> | 1 | 0
953538a96d5f4e0586e2d8fafe939193 | 127.12.104.194:40755 | HEALTHY | <none> | 0 | 0
c721bd643fa948efa627bf307e05b28c | 127.12.104.193:35911 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260924 15:09:15.905009 12707 log_verifier.cc:126] Checking tablet 2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:15.966879 12707 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 2d757ecbc6d245d19d1cd8575d3d0651
I20260924 15:09:15.970501 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:15.998075 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:15.998862 12707 tablet_replica.cc:333] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c: stopping tablet replica
I20260924 15:09:15.999404 12707 raft_consensus.cc:2243] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:15.999857 12707 raft_consensus.cc:2272] T 2d757ecbc6d245d19d1cd8575d3d0651 P c721bd643fa948efa627bf307e05b28c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:16.021204 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:16.032727 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:09:16.057315 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:16.058177 12707 tablet_replica.cc:333] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193: stopping tablet replica
I20260924 15:09:16.058853 12707 raft_consensus.cc:2243] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:16.059453 12707 raft_consensus.cc:2272] T 2d757ecbc6d245d19d1cd8575d3d0651 P 953538a96d5f4e0586e2d8fafe939193 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:16.080811 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:09:16.093149 12707 tablet_server.cc:179] TabletServer@127.12.104.195:0 shutting down...
W20260924 15:09:16.121743 16464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.104.194:40755: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260924 15:09:16.121447 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:16.123638 12707 tablet_replica.cc:333] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23: stopping tablet replica
I20260924 15:09:16.124622 12707 raft_consensus.cc:2243] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 LEADER]: Raft consensus shutting down.
I20260924 15:09:16.125344 12707 raft_consensus.cc:2272] T 2d757ecbc6d245d19d1cd8575d3d0651 P 9035a5477f7c4ed583fb09f0982aef23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:16.148782 12707 tablet_server.cc:196] TabletServer@127.12.104.195:0 shutdown complete.
I20260924 15:09:16.161212 12707 master.cc:562] Master@127.12.104.254:38479 shutting down...
I20260924 15:09:16.183652 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:16.184223 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:16.184580 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc8cc8a8e5ca4e22b69c2e5605998ed4: stopping tablet replica
I20260924 15:09:16.203799 12707 master.cc:584] Master@127.12.104.254:38479 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (6129 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260924 15:09:16.237401 12707 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.104.254:46379
I20260924 15:09:16.238605 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:16.244835 16788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.245290 16789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.246488 16791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.246842 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:16.247790 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:16.248021 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:16.248183 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262556248160 us; error 0 us; skew 500 ppm
I20260924 15:09:16.248697 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:16.251120 12707 webserver.cc:533] Webserver started at http://127.12.104.254:43111/ using document root <none> and password file <none>
I20260924 15:09:16.251616 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:16.251818 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:16.252100 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:16.253225 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/master-0-root/instance:
uuid: "58f5f7456f7545bc9566363a004d07a0"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.257869 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260924 15:09:16.261409 16796 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.262267 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260924 15:09:16.262655 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/master-0-root
uuid: "58f5f7456f7545bc9566363a004d07a0"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.262974 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:16.283959 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:16.285221 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:16.330119 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.254:46379
I20260924 15:09:16.330207 16847 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.254:46379 every 8 connection(s)
I20260924 15:09:16.334251 16848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:16.345546 16848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0: Bootstrap starting.
I20260924 15:09:16.350633 16848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:16.355105 16848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0: No bootstrap required, opened a new log
I20260924 15:09:16.357311 16848 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER }
I20260924 15:09:16.357719 16848 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:16.357970 16848 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f5f7456f7545bc9566363a004d07a0, State: Initialized, Role: FOLLOWER
I20260924 15:09:16.358691 16848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER }
I20260924 15:09:16.359273 16848 raft_consensus.cc:399] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 15:09:16.359546 16848 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 15:09:16.359908 16848 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:16.367223 16848 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER }
I20260924 15:09:16.367798 16848 leader_election.cc:304] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58f5f7456f7545bc9566363a004d07a0; no voters:
I20260924 15:09:16.368964 16848 leader_election.cc:290] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 15:09:16.369444 16851 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:16.370791 16851 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 1 LEADER]: Becoming Leader. State: Replica: 58f5f7456f7545bc9566363a004d07a0, State: Running, Role: LEADER
I20260924 15:09:16.371490 16851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER }
I20260924 15:09:16.371907 16848 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 15:09:16.374272 16852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58f5f7456f7545bc9566363a004d07a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER } }
I20260924 15:09:16.375015 16852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:16.374276 16853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f5f7456f7545bc9566363a004d07a0. Latest consensus state: current_term: 1 leader_uuid: "58f5f7456f7545bc9566363a004d07a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f5f7456f7545bc9566363a004d07a0" member_type: VOTER } }
I20260924 15:09:16.376153 16853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [sys.catalog]: This master's current role is: LEADER
I20260924 15:09:16.386796 16857 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260924 15:09:16.387198 12707 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 15:09:16.391696 16857 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260924 15:09:16.400810 16857 catalog_manager.cc:1396] Generated new cluster ID: 1e6f987cc8b648c2b16e79ea36d76a31
I20260924 15:09:16.401160 16857 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260924 15:09:16.413704 16857 catalog_manager.cc:1419] Generated new certificate authority record
I20260924 15:09:16.415050 16857 catalog_manager.cc:1553] Loading token signing keys...
I20260924 15:09:16.427390 16857 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0: Generated new TSK 0
I20260924 15:09:16.428035 16857 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260924 15:09:16.453719 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:16.460033 16869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.461453 16870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.462983 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:16.462955 16872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.464248 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:16.464510 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:16.464685 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262556464667 us; error 0 us; skew 500 ppm
I20260924 15:09:16.465286 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:16.467669 12707 webserver.cc:533] Webserver started at http://127.12.104.193:44077/ using document root <none> and password file <none>
I20260924 15:09:16.468163 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:16.468379 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:16.468660 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:16.469755 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-0-root/instance:
uuid: "529515e1e86540dfa86b40f274ec05f4"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.474597 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 15:09:16.477988 16877 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.478833 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 15:09:16.479161 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-0-root
uuid: "529515e1e86540dfa86b40f274ec05f4"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.479447 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:16.497936 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:16.499277 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:16.500733 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:16.503145 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:16.503369 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.503628 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:16.503799 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.546286 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.193:35187
I20260924 15:09:16.546453 16939 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.193:35187 every 8 connection(s)
I20260924 15:09:16.551851 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:16.558909 16944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.561443 16945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.562778 12707 server_base.cc:1061] running on GCE node
W20260924 15:09:16.563023 16947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.564500 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:16.564880 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:16.565133 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262556565111 us; error 0 us; skew 500 ppm
I20260924 15:09:16.566001 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:16.567608 16940 heartbeater.cc:344] Connected to a master server at 127.12.104.254:46379
I20260924 15:09:16.568111 16940 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:16.569121 16940 heartbeater.cc:507] Master 127.12.104.254:46379 requested a full tablet report, sending...
I20260924 15:09:16.569674 12707 webserver.cc:533] Webserver started at http://127.12.104.194:42575/ using document root <none> and password file <none>
I20260924 15:09:16.570443 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:16.570667 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:16.571022 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:16.572069 16813 ts_manager.cc:194] Registered new tserver with Master: 529515e1e86540dfa86b40f274ec05f4 (127.12.104.193:35187)
I20260924 15:09:16.572450 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-1-root/instance:
uuid: "26a4dcbbe4f444269359767f124c4b15"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.574164 16813 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60714
I20260924 15:09:16.577944 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 15:09:16.582571 16952 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.583451 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:16.583789 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-1-root
uuid: "26a4dcbbe4f444269359767f124c4b15"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.584132 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:16.602768 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:16.604044 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:16.605623 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:16.608013 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:16.608268 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.608541 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:16.608726 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.651724 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.194:35731
I20260924 15:09:16.651855 17014 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.194:35731 every 8 connection(s)
I20260924 15:09:16.656713 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:16.664620 17018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.666183 17019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.672443 17015 heartbeater.cc:344] Connected to a master server at 127.12.104.254:46379
I20260924 15:09:16.672993 17015 heartbeater.cc:461] Registering TS with master...
W20260924 15:09:16.673529 17021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.674085 17015 heartbeater.cc:507] Master 127.12.104.254:46379 requested a full tablet report, sending...
I20260924 15:09:16.674917 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:16.676039 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 15:09:16.676231 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:16.676452 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262556676418 us; error 0 us; skew 500 ppm
I20260924 15:09:16.676653 16813 ts_manager.cc:194] Registered new tserver with Master: 26a4dcbbe4f444269359767f124c4b15 (127.12.104.194:35731)
I20260924 15:09:16.677099 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:16.678526 16813 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60728
I20260924 15:09:16.680059 12707 webserver.cc:533] Webserver started at http://127.12.104.195:38829/ using document root <none> and password file <none>
I20260924 15:09:16.680788 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:16.681079 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:16.681460 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:16.682651 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-2-root/instance:
uuid: "6da2e0601a67404b9d344c79db64ceef"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.687300 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260924 15:09:16.690940 17026 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.691787 12707 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260924 15:09:16.692118 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-2-root
uuid: "6da2e0601a67404b9d344c79db64ceef"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.692431 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:16.725914 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:16.727290 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:16.728809 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:16.731205 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:16.731453 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.731704 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:16.731886 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.773404 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.195:39757
I20260924 15:09:16.773542 17088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.195:39757 every 8 connection(s)
I20260924 15:09:16.778644 12707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260924 15:09:16.788177 17092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.788308 17093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 15:09:16.791178 17095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 15:09:16.791980 17089 heartbeater.cc:344] Connected to a master server at 127.12.104.254:46379
I20260924 15:09:16.792347 12707 server_base.cc:1061] running on GCE node
I20260924 15:09:16.792409 17089 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:16.793560 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260924 15:09:16.793632 17089 heartbeater.cc:507] Master 127.12.104.254:46379 requested a full tablet report, sending...
W20260924 15:09:16.793954 12707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 15:09:16.794324 12707 hybrid_clock.cc:648] HybridClock initialized: now 1790262556794278 us; error 0 us; skew 500 ppm
I20260924 15:09:16.795039 12707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 15:09:16.796056 16813 ts_manager.cc:194] Registered new tserver with Master: 6da2e0601a67404b9d344c79db64ceef (127.12.104.195:39757)
I20260924 15:09:16.797617 16813 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60730
I20260924 15:09:16.797880 12707 webserver.cc:533] Webserver started at http://127.12.104.196:46447/ using document root <none> and password file <none>
I20260924 15:09:16.798555 12707 fs_manager.cc:362] Metadata directory not provided
I20260924 15:09:16.798786 12707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 15:09:16.799115 12707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 15:09:16.800685 12707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root/instance:
uuid: "d97323095ca545a8a8558aeb75ae9dfe"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.805775 12707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 15:09:16.809484 17100 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.810501 12707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 15:09:16.810894 12707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root
uuid: "d97323095ca545a8a8558aeb75ae9dfe"
format_stamp: "Formatted at 2026-09-24 15:09:16 on dist-test-slave-9164"
I20260924 15:09:16.811163 12707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 15:09:16.841720 12707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260924 15:09:16.843235 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 15:09:16.844908 12707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260924 15:09:16.847707 12707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 15:09:16.847965 12707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.848253 12707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260924 15:09:16.848441 12707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 15:09:16.904062 12707 rpc_server.cc:307] RPC server started. Bound to: 127.12.104.196:40105
I20260924 15:09:16.904213 17162 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.104.196:40105 every 8 connection(s)
I20260924 15:09:16.925105 17163 heartbeater.cc:344] Connected to a master server at 127.12.104.254:46379
I20260924 15:09:16.925742 17163 heartbeater.cc:461] Registering TS with master...
I20260924 15:09:16.926720 17163 heartbeater.cc:507] Master 127.12.104.254:46379 requested a full tablet report, sending...
I20260924 15:09:16.929492 16813 ts_manager.cc:194] Registered new tserver with Master: d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105)
I20260924 15:09:16.929662 12707 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.020472601s
I20260924 15:09:16.931504 16813 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60734
I20260924 15:09:16.979337 16813 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60752:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 15:09:17.035367 16905 tablet_service.cc:1511] Processing CreateTablet for tablet c1d251c00bff47c4a3c3a92200b62f70 (DEFAULT_TABLE table=test-workload [id=d7ec5da5ebca4f6e925e2c43eb942c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:17.036777 16905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1d251c00bff47c4a3c3a92200b62f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:17.038691 17054 tablet_service.cc:1511] Processing CreateTablet for tablet c1d251c00bff47c4a3c3a92200b62f70 (DEFAULT_TABLE table=test-workload [id=d7ec5da5ebca4f6e925e2c43eb942c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:17.040062 17054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1d251c00bff47c4a3c3a92200b62f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:17.043180 17128 tablet_service.cc:1511] Processing CreateTablet for tablet c1d251c00bff47c4a3c3a92200b62f70 (DEFAULT_TABLE table=test-workload [id=d7ec5da5ebca4f6e925e2c43eb942c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 15:09:17.044559 17128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1d251c00bff47c4a3c3a92200b62f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:17.054864 17186 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: Bootstrap starting.
I20260924 15:09:17.057219 17187 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: Bootstrap starting.
I20260924 15:09:17.062289 17186 tablet_bootstrap.cc:654] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:17.065893 17187 tablet_bootstrap.cc:654] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:17.072062 17188 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Bootstrap starting.
I20260924 15:09:17.076757 17187 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: No bootstrap required, opened a new log
I20260924 15:09:17.077307 17187 ts_tablet_manager.cc:1403] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.007s
I20260924 15:09:17.080224 17187 raft_consensus.cc:359] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.080994 17186 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: No bootstrap required, opened a new log
I20260924 15:09:17.081100 17187 raft_consensus.cc:385] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:17.081586 17188 tablet_bootstrap.cc:654] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Neither blocks nor log segments found. Creating new log.
I20260924 15:09:17.081688 17186 ts_tablet_manager.cc:1403] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: Time spent bootstrapping tablet: real 0.027s user 0.017s sys 0.007s
I20260924 15:09:17.081632 17187 raft_consensus.cc:740] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da2e0601a67404b9d344c79db64ceef, State: Initialized, Role: FOLLOWER
I20260924 15:09:17.082824 17187 consensus_queue.cc:260] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.085716 17186 raft_consensus.cc:359] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.086922 17186 raft_consensus.cc:385] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:17.087332 17186 raft_consensus.cc:740] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529515e1e86540dfa86b40f274ec05f4, State: Initialized, Role: FOLLOWER
I20260924 15:09:17.088331 17186 consensus_queue.cc:260] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.091712 16940 heartbeater.cc:499] Master 127.12.104.254:46379 was elected leader, sending a full tablet report...
I20260924 15:09:17.092178 17186 ts_tablet_manager.cc:1434] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260924 15:09:17.094600 17187 ts_tablet_manager.cc:1434] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: Time spent starting tablet: real 0.017s user 0.013s sys 0.006s
I20260924 15:09:17.094822 17089 heartbeater.cc:499] Master 127.12.104.254:46379 was elected leader, sending a full tablet report...
I20260924 15:09:17.096963 17188 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: No bootstrap required, opened a new log
I20260924 15:09:17.097509 17188 ts_tablet_manager.cc:1403] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Time spent bootstrapping tablet: real 0.026s user 0.017s sys 0.004s
I20260924 15:09:17.100665 17188 raft_consensus.cc:359] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.101432 17188 raft_consensus.cc:385] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 15:09:17.101727 17188 raft_consensus.cc:740] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97323095ca545a8a8558aeb75ae9dfe, State: Initialized, Role: FOLLOWER
I20260924 15:09:17.102670 17188 consensus_queue.cc:260] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.105260 17163 heartbeater.cc:499] Master 127.12.104.254:46379 was elected leader, sending a full tablet report...
I20260924 15:09:17.105243 17193 raft_consensus.cc:493] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 15:09:17.106107 17188 ts_tablet_manager.cc:1434] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Time spent starting tablet: real 0.008s user 0.009s sys 0.001s
I20260924 15:09:17.106266 17193 raft_consensus.cc:515] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.108840 17193 leader_election.cc:290] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105), 6da2e0601a67404b9d344c79db64ceef (127.12.104.195:39757)
I20260924 15:09:17.122750 17138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1d251c00bff47c4a3c3a92200b62f70" candidate_uuid: "529515e1e86540dfa86b40f274ec05f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97323095ca545a8a8558aeb75ae9dfe" is_pre_election: true
I20260924 15:09:17.123508 17138 raft_consensus.cc:2468] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529515e1e86540dfa86b40f274ec05f4 in term 0.
I20260924 15:09:17.124740 16878 leader_election.cc:304] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529515e1e86540dfa86b40f274ec05f4, d97323095ca545a8a8558aeb75ae9dfe; no voters:
I20260924 15:09:17.125589 17193 raft_consensus.cc:2804] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 15:09:17.125967 17193 raft_consensus.cc:493] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 15:09:17.126295 17193 raft_consensus.cc:3060] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:17.129088 17064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1d251c00bff47c4a3c3a92200b62f70" candidate_uuid: "529515e1e86540dfa86b40f274ec05f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da2e0601a67404b9d344c79db64ceef" is_pre_election: true
I20260924 15:09:17.129760 17064 raft_consensus.cc:2468] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529515e1e86540dfa86b40f274ec05f4 in term 0.
I20260924 15:09:17.132622 17193 raft_consensus.cc:515] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.134585 17193 leader_election.cc:290] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [CANDIDATE]: Term 1 election: Requested vote from peers d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105), 6da2e0601a67404b9d344c79db64ceef (127.12.104.195:39757)
I20260924 15:09:17.135365 17138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1d251c00bff47c4a3c3a92200b62f70" candidate_uuid: "529515e1e86540dfa86b40f274ec05f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97323095ca545a8a8558aeb75ae9dfe"
I20260924 15:09:17.135620 17064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1d251c00bff47c4a3c3a92200b62f70" candidate_uuid: "529515e1e86540dfa86b40f274ec05f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da2e0601a67404b9d344c79db64ceef"
I20260924 15:09:17.135880 17138 raft_consensus.cc:3060] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:17.136179 17064 raft_consensus.cc:3060] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 0 FOLLOWER]: Advancing to term 1
I20260924 15:09:17.141580 17138 raft_consensus.cc:2468] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529515e1e86540dfa86b40f274ec05f4 in term 1.
I20260924 15:09:17.141710 17064 raft_consensus.cc:2468] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529515e1e86540dfa86b40f274ec05f4 in term 1.
I20260924 15:09:17.142906 16878 leader_election.cc:304] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529515e1e86540dfa86b40f274ec05f4, d97323095ca545a8a8558aeb75ae9dfe; no voters:
I20260924 15:09:17.143618 17193 raft_consensus.cc:2804] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 15:09:17.144716 17193 raft_consensus.cc:697] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 LEADER]: Becoming Leader. State: Replica: 529515e1e86540dfa86b40f274ec05f4, State: Running, Role: LEADER
I20260924 15:09:17.145558 17193 consensus_queue.cc:237] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:17.152658 16812 catalog_manager.cc:5838] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 529515e1e86540dfa86b40f274ec05f4 (127.12.104.193). New cstate: current_term: 1 leader_uuid: "529515e1e86540dfa86b40f274ec05f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } health_report { overall_health: UNKNOWN } } }
I20260924 15:09:17.421381 17054 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:17.424846 16905 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:17.427188 16980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:17.432471 17128 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:17.681641 17015 heartbeater.cc:499] Master 127.12.104.254:46379 was elected leader, sending a full tablet report...
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
58f5f7456f7545bc9566363a004d07a0 | 127.12.104.254:46379 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | HEALTHY | <none> | 0 | 0
529515e1e86540dfa86b40f274ec05f4 | 127.12.104.193:35187 | HEALTHY | <none> | 1 | 0
6da2e0601a67404b9d344c79db64ceef | 127.12.104.195:39757 | HEALTHY | <none> | 0 | 0
d97323095ca545a8a8558aeb75ae9dfe | 127.12.104.196:40105 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet c1d251c00bff47c4a3c3a92200b62f70 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
529515e1e86540dfa86b40f274ec05f4 (127.12.104.193:35187): RUNNING [LEADER]
d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105): RUNNING
6da2e0601a67404b9d344c79db64ceef (127.12.104.195:39757): RUNNING
All reported replicas are:
A = 529515e1e86540dfa86b40f274ec05f4
B = d97323095ca545a8a8558aeb75ae9dfe
C = 6da2e0601a67404b9d344c79db64ceef
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | 26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:17.720757 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:17.737447 17229 consensus_queue.cc:1048] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97323095ca545a8a8558aeb75ae9dfe" member_type: VOTER last_known_addr { host: "127.12.104.196" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:09:17.754923 17229 consensus_queue.cc:1048] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 15:09:18.114537 16980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:18.126499 17054 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:18.126935 16905 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:18.132311 17128 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
58f5f7456f7545bc9566363a004d07a0 | 127.12.104.254:46379 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | HEALTHY | <none> | 0 | 0
529515e1e86540dfa86b40f274ec05f4 | 127.12.104.193:35187 | HEALTHY | <none> | 1 | 0
6da2e0601a67404b9d344c79db64ceef | 127.12.104.195:39757 | HEALTHY | <none> | 0 | 0
d97323095ca545a8a8558aeb75ae9dfe | 127.12.104.196:40105 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | 26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260924 15:09:18.421604 12707 log_verifier.cc:126] Checking tablet c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:18.491585 12707 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:18.492210 12707 tablet_replica.cc:333] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: stopping tablet replica
I20260924 15:09:18.492806 12707 raft_consensus.cc:2243] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:18.493330 12707 raft_consensus.cc:2272] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:18.746738 16905 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:18.754087 17054 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:18.761632 16980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:18.761885 17128 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:18.920512 16878 consensus_queue.cc:579] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: The tablet replica hosted on peer d97323095ca545a8a8558aeb75ae9dfe has failed
W20260924 15:09:18.920982 16878 consensus_peers.cc:597] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 -> Peer d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105): Couldn't send request to peer d97323095ca545a8a8558aeb75ae9dfe. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:18.949046 16915 consensus_queue.cc:237] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } }
I20260924 15:09:18.976924 17064 raft_consensus.cc:1275] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Refusing update from remote peer 529515e1e86540dfa86b40f274ec05f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 15:09:18.978843 17301 consensus_queue.cc:1048] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 15:09:18.988672 17269 raft_consensus.cc:2955] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } }
I20260924 15:09:18.990782 17064 raft_consensus.cc:2955] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } }
I20260924 15:09:18.998979 16797 catalog_manager.cc:5245] ChangeConfig:REMOVE_PEER RPC for tablet c1d251c00bff47c4a3c3a92200b62f70 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 15:09:19.003477 16813 catalog_manager.cc:5838] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 reported cstate change: config changed from index -1 to 2, VOTER d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196) evicted. New cstate: current_term: 1 leader_uuid: "529515e1e86540dfa86b40f274ec05f4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } health_report { overall_health: HEALTHY } } }
I20260924 15:09:19.021291 17128 tablet_service.cc:1558] Processing DeleteTablet for tablet c1d251c00bff47c4a3c3a92200b62f70 with delete_type TABLET_DATA_TOMBSTONED (TS d97323095ca545a8a8558aeb75ae9dfe not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:33032
I20260924 15:09:19.023228 17309 ts_tablet_manager.cc:1916] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 15:09:19.025537 16915 consensus_queue.cc:237] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } }
I20260924 15:09:19.037185 17064 raft_consensus.cc:1275] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Refusing update from remote peer 529515e1e86540dfa86b40f274ec05f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 15:09:19.040524 17309 ts_tablet_manager.cc:1929] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260924 15:09:19.041026 17309 log.cc:1199] T c1d251c00bff47c4a3c3a92200b62f70 P d97323095ca545a8a8558aeb75ae9dfe: Deleting WAL directory at /tmp/dist-test-task0vYK5N/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790262492846473-12707-0/minicluster-data/ts-3-root/wals/c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:19.041354 17301 consensus_queue.cc:1048] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 15:09:19.043026 16797 catalog_manager.cc:5063] TS d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105): tablet c1d251c00bff47c4a3c3a92200b62f70 (table test-workload [id=d7ec5da5ebca4f6e925e2c43eb942c81]) successfully deleted
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
58f5f7456f7545bc9566363a004d07a0 | 127.12.104.254:46379 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | HEALTHY | <none> | 0 | 0
529515e1e86540dfa86b40f274ec05f4 | 127.12.104.193:35187 | HEALTHY | <none> | 1 | 0
6da2e0601a67404b9d344c79db64ceef | 127.12.104.195:39757 | HEALTHY | <none> | 0 | 0
d97323095ca545a8a8558aeb75ae9dfe | 127.12.104.196:40105 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet c1d251c00bff47c4a3c3a92200b62f70 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
529515e1e86540dfa86b40f274ec05f4 (127.12.104.193:35187): RUNNING [LEADER]
d97323095ca545a8a8558aeb75ae9dfe (127.12.104.196:40105): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
6da2e0601a67404b9d344c79db64ceef (127.12.104.195:39757): RUNNING
All reported replicas are:
A = 529515e1e86540dfa86b40f274ec05f4
B = d97323095ca545a8a8558aeb75ae9dfe
C = 6da2e0601a67404b9d344c79db64ceef
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | [config not available] | | |
C | A* B C | 1 | -1 | Yes
I20260924 15:09:19.049942 17269 raft_consensus.cc:2955] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 26a4dcbbe4f444269359767f124c4b15 (127.12.104.194) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } } }
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | 26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | 0
Total Count Summary
| Total Count
----------------+-------------
I20260924 15:09:19.052039 17064 raft_consensus.cc:2955] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 26a4dcbbe4f444269359767f124c4b15 (127.12.104.194) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } } }
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260924 15:09:19.062352 16797 catalog_manager.cc:5245] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c1d251c00bff47c4a3c3a92200b62f70 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20260924 15:09:19.063488 16878 consensus_peers.cc:597] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 -> Peer 26a4dcbbe4f444269359767f124c4b15 (127.12.104.194:35731): Couldn't send request to peer 26a4dcbbe4f444269359767f124c4b15. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1d251c00bff47c4a3c3a92200b62f70. This is attempt 1: this message will repeat every 5th retry.
I20260924 15:09:19.064890 12707 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260924 15:09:19.070645 16813 catalog_manager.cc:5838] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef reported cstate change: config changed from index 2 to 3, NON_VOTER 26a4dcbbe4f444269359767f124c4b15 (127.12.104.194) added. New cstate: current_term: 1 leader_uuid: "529515e1e86540dfa86b40f274ec05f4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } } }
I20260924 15:09:19.451483 17338 ts_tablet_manager.cc:933] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Initiating tablet copy from peer 529515e1e86540dfa86b40f274ec05f4 (127.12.104.193:35187)
I20260924 15:09:19.472607 16980 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:19.475136 17338 tablet_copy_client.cc:323] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: tablet copy: Beginning tablet copy session from remote peer at address 127.12.104.193:35187
I20260924 15:09:19.476420 17054 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:19.477538 16905 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260924 15:09:19.489284 16925 tablet_copy_service.cc:140] P 529515e1e86540dfa86b40f274ec05f4: Received BeginTabletCopySession request for tablet c1d251c00bff47c4a3c3a92200b62f70 from peer 26a4dcbbe4f444269359767f124c4b15 ({username='slave'} at 127.0.0.1:33774)
I20260924 15:09:19.490041 16925 tablet_copy_service.cc:161] P 529515e1e86540dfa86b40f274ec05f4: Beginning new tablet copy session on tablet c1d251c00bff47c4a3c3a92200b62f70 from peer 26a4dcbbe4f444269359767f124c4b15 at {username='slave'} at 127.0.0.1:33774: session id = 26a4dcbbe4f444269359767f124c4b15-c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:19.492427 17128 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260924 15:09:19.497701 16925 tablet_copy_source_session.cc:215] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 15:09:19.501099 17338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1d251c00bff47c4a3c3a92200b62f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 15:09:19.519624 17338 tablet_copy_client.cc:806] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: tablet copy: Starting download of 0 data blocks...
I20260924 15:09:19.520437 17338 tablet_copy_client.cc:670] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: tablet copy: Starting download of 1 WAL segments...
I20260924 15:09:19.527386 17338 tablet_copy_client.cc:538] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 15:09:19.537720 17338 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Bootstrap starting.
I20260924 15:09:19.574347 17338 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 15:09:19.575600 17338 tablet_bootstrap.cc:492] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Bootstrap complete.
I20260924 15:09:19.576434 17338 ts_tablet_manager.cc:1403] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Time spent bootstrapping tablet: real 0.039s user 0.032s sys 0.008s
I20260924 15:09:19.579686 17338 raft_consensus.cc:359] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } }
I20260924 15:09:19.580559 17338 raft_consensus.cc:740] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 26a4dcbbe4f444269359767f124c4b15, State: Initialized, Role: LEARNER
I20260924 15:09:19.581354 17338 consensus_queue.cc:260] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "529515e1e86540dfa86b40f274ec05f4" member_type: VOTER last_known_addr { host: "127.12.104.193" port: 35187 } } peers { permanent_uuid: "6da2e0601a67404b9d344c79db64ceef" member_type: VOTER last_known_addr { host: "127.12.104.195" port: 39757 } } peers { permanent_uuid: "26a4dcbbe4f444269359767f124c4b15" member_type: NON_VOTER last_known_addr { host: "127.12.104.194" port: 35731 } attrs { promote: true } }
I20260924 15:09:19.596019 17338 ts_tablet_manager.cc:1434] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: Time spent starting tablet: real 0.019s user 0.006s sys 0.011s
I20260924 15:09:19.598034 16925 tablet_copy_service.cc:342] P 529515e1e86540dfa86b40f274ec05f4: Request end of tablet copy session 26a4dcbbe4f444269359767f124c4b15-c1d251c00bff47c4a3c3a92200b62f70 received from {username='slave'} at 127.0.0.1:33774
I20260924 15:09:19.598510 16925 tablet_copy_service.cc:434] P 529515e1e86540dfa86b40f274ec05f4: ending tablet copy session 26a4dcbbe4f444269359767f124c4b15-c1d251c00bff47c4a3c3a92200b62f70 on tablet c1d251c00bff47c4a3c3a92200b62f70 with peer 26a4dcbbe4f444269359767f124c4b15
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
58f5f7456f7545bc9566363a004d07a0 | 127.12.104.254:46379 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
26a4dcbbe4f444269359767f124c4b15 | 127.12.104.194:35731 | HEALTHY | <none> | 0 | 0
529515e1e86540dfa86b40f274ec05f4 | 127.12.104.193:35187 | HEALTHY | <none> | 1 | 0
6da2e0601a67404b9d344c79db64ceef | 127.12.104.195:39757 | HEALTHY | <none> | 0 | 0
d97323095ca545a8a8558aeb75ae9dfe | 127.12.104.196:40105 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d97323095ca545a8a8558aeb75ae9dfe | 127.12.104.196:40105 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260924 15:09:19.800043 12707 log_verifier.cc:126] Checking tablet c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:19.860605 12707 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c1d251c00bff47c4a3c3a92200b62f70
I20260924 15:09:19.862813 12707 tablet_server.cc:179] TabletServer@127.12.104.193:0 shutting down...
I20260924 15:09:19.885708 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:19.886449 12707 tablet_replica.cc:333] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4: stopping tablet replica
I20260924 15:09:19.886989 12707 raft_consensus.cc:2243] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:19.887862 12707 raft_consensus.cc:2272] T c1d251c00bff47c4a3c3a92200b62f70 P 529515e1e86540dfa86b40f274ec05f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:19.908831 12707 tablet_server.cc:196] TabletServer@127.12.104.193:0 shutdown complete.
I20260924 15:09:19.919610 12707 tablet_server.cc:179] TabletServer@127.12.104.194:0 shutting down...
I20260924 15:09:19.940282 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:19.941208 12707 tablet_replica.cc:333] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15: stopping tablet replica
I20260924 15:09:19.941951 12707 raft_consensus.cc:2243] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15 [term 1 LEARNER]: Raft consensus shutting down.
I20260924 15:09:19.942456 12707 raft_consensus.cc:2272] T c1d251c00bff47c4a3c3a92200b62f70 P 26a4dcbbe4f444269359767f124c4b15 [term 1 LEARNER]: Raft consensus is shut down!
I20260924 15:09:19.962610 12707 tablet_server.cc:196] TabletServer@127.12.104.194:0 shutdown complete.
I20260924 15:09:19.972858 12707 tablet_server.cc:179] TabletServer@127.12.104.195:0 shutting down...
I20260924 15:09:19.991992 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:19.992651 12707 tablet_replica.cc:333] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef: stopping tablet replica
I20260924 15:09:19.993176 12707 raft_consensus.cc:2243] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 15:09:19.993710 12707 raft_consensus.cc:2272] T c1d251c00bff47c4a3c3a92200b62f70 P 6da2e0601a67404b9d344c79db64ceef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:20.014602 12707 tablet_server.cc:196] TabletServer@127.12.104.195:0 shutdown complete.
I20260924 15:09:20.025008 12707 tablet_server.cc:179] TabletServer@127.12.104.196:0 shutting down...
I20260924 15:09:20.044049 12707 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260924 15:09:20.061878 12707 tablet_server.cc:196] TabletServer@127.12.104.196:0 shutdown complete.
I20260924 15:09:20.071780 12707 master.cc:562] Master@127.12.104.254:46379 shutting down...
I20260924 15:09:20.088871 12707 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 15:09:20.089406 12707 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 15:09:20.089668 12707 tablet_replica.cc:333] T 00000000000000000000000000000000 P 58f5f7456f7545bc9566363a004d07a0: stopping tablet replica
I20260924 15:09:20.107343 12707 master.cc:584] Master@127.12.104.254:46379 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (3901 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (10030 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (67108 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260924 15:09:20.145440 12707 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~11 seconds ago
I20260924 15:09:20.145712 12707 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 1 messages since previous log ~4 seconds ago
I20260924 15:09:20.145862 12707 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 76 messages since previous log ~58 seconds ago
I20260924 15:09:20.146111 12707 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2433: suppressed but not reported on 7 messages since previous log ~14 seconds ago