[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 00:03:12.882889 5521 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.163s user 0.406s sys 0.754s
W20260810 00:03:12.883484 5521 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.164s user 0.406s sys 0.754s
I20260810 00:03:12.886422 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:44703
I20260810 00:03:12.892455 5521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 00:03:12.893882 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:12.911872 5533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:12.915261 5535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:12.913271 5532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:13.014708 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:13.015901 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:13.016186 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:13.016364 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320193016344 us; error 0 us; skew 500 ppm
I20260810 00:03:13.017076 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:13.025933 5521 webserver.cc:460] Webserver started at http://127.5.100.126:39105/ using document root <none> and password file <none>
I20260810 00:03:13.026811 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:13.026993 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:13.027381 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:13.033203 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "7d0a134d5c8744f0be216b16f2551e98"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.040926 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260810 00:03:13.046875 5541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.048018 5521 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:03:13.048446 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "7d0a134d5c8744f0be216b16f2551e98"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.048791 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:13.074890 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:13.076472 5521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 00:03:13.076948 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:13.149413 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:44703
I20260810 00:03:13.149506 5594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:44703 every 8 connection(s)
I20260810 00:03:13.155560 5596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:13.179490 5596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: Bootstrap starting.
I20260810 00:03:13.185681 5596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:13.187455 5596 log.cc:824] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: Log is configured to *not* fsync() on all Append() calls
I20260810 00:03:13.191668 5596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: No bootstrap required, opened a new log
I20260810 00:03:13.213649 5596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER }
I20260810 00:03:13.214265 5596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:13.214550 5596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0a134d5c8744f0be216b16f2551e98, State: Initialized, Role: FOLLOWER
I20260810 00:03:13.215314 5596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER }
I20260810 00:03:13.215829 5596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:13.216146 5596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:13.216471 5596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:13.220374 5596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER }
I20260810 00:03:13.221058 5596 leader_election.cc:302] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d0a134d5c8744f0be216b16f2551e98; no voters:
I20260810 00:03:13.222815 5596 leader_election.cc:288] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:13.223234 5599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:13.225906 5599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 1 LEADER]: Becoming Leader. State: Replica: 7d0a134d5c8744f0be216b16f2551e98, State: Running, Role: LEADER
I20260810 00:03:13.226704 5599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER }
I20260810 00:03:13.227710 5596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:13.239852 5600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d0a134d5c8744f0be216b16f2551e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER } }
I20260810 00:03:13.240159 5601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d0a134d5c8744f0be216b16f2551e98. Latest consensus state: current_term: 1 leader_uuid: "7d0a134d5c8744f0be216b16f2551e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0a134d5c8744f0be216b16f2551e98" member_type: VOTER } }
I20260810 00:03:13.240590 5600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:13.240733 5601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:13.250813 5610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:13.263475 5610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:13.279614 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:13.288666 5610 catalog_manager.cc:1347] Generated new cluster ID: f922a1b2a9c640e6bc74f927acbdb27f
I20260810 00:03:13.288969 5610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:13.345171 5610 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:13.346946 5610 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:13.360106 5610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: Generated new TSK 0
I20260810 00:03:13.361281 5610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:13.417299 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:13.425709 5618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:13.427423 5619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:13.432945 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:13.433373 5621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:13.434257 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:13.434441 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:13.434648 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320193434614 us; error 0 us; skew 500 ppm
I20260810 00:03:13.435258 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:13.439910 5521 webserver.cc:460] Webserver started at http://127.5.100.65:41863/ using document root <none> and password file <none>
I20260810 00:03:13.440474 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:13.440680 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:13.440965 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:13.442245 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "7adc7b883bbd4f8ea378411cddf76869"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.448187 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 00:03:13.453163 5628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.453969 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260810 00:03:13.454236 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "7adc7b883bbd4f8ea378411cddf76869"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.454502 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:13.481892 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:13.483183 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:13.485991 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:13.490593 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:13.490911 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.491209 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:13.491371 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.600217 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:43477
I20260810 00:03:13.600752 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:43477 every 8 connection(s)
I20260810 00:03:13.607179 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:13.619406 5703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:13.621179 5704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:13.627033 5706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:13.633504 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:13.634495 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:13.634801 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:13.635001 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320193634984 us; error 0 us; skew 500 ppm
I20260810 00:03:13.635578 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:13.638528 5521 webserver.cc:460] Webserver started at http://127.5.100.66:34527/ using document root <none> and password file <none>
I20260810 00:03:13.639170 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:13.639415 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:13.639741 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:13.641291 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "f5af311658ce4377a6d60edbba6ca855"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.644236 5698 heartbeater.cc:347] Connected to a master server at 127.5.100.126:44703
I20260810 00:03:13.644692 5698 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:13.645713 5698 heartbeater.cc:511] Master 127.5.100.126:44703 requested a full tablet report, sending...
I20260810 00:03:13.648511 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 00:03:13.649251 5558 ts_manager.cc:194] Registered new tserver with Master: 7adc7b883bbd4f8ea378411cddf76869 (127.5.100.65:43477)
I20260810 00:03:13.652355 5558 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48650
I20260810 00:03:13.654220 5712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.655160 5521 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 00:03:13.655529 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "f5af311658ce4377a6d60edbba6ca855"
format_stamp: "Formatted at 2026-08-10 00:03:13 on dist-test-slave-8w9v"
I20260810 00:03:13.655908 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:13.668936 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:13.670040 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:13.672199 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:13.675293 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:13.675493 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.675730 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:13.675895 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:13.757135 5698 heartbeater.cc:503] Master 127.5.100.126:44703 was elected leader, sending a full tablet report...
I20260810 00:03:13.777801 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:39377
I20260810 00:03:13.777915 5779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:39377 every 8 connection(s)
I20260810 00:03:13.795711 5780 heartbeater.cc:347] Connected to a master server at 127.5.100.126:44703
I20260810 00:03:13.796120 5780 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:13.796871 5780 heartbeater.cc:511] Master 127.5.100.126:44703 requested a full tablet report, sending...
I20260810 00:03:13.798738 5558 ts_manager.cc:194] Registered new tserver with Master: f5af311658ce4377a6d60edbba6ca855 (127.5.100.66:39377)
I20260810 00:03:13.799762 5521 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.018272672s
I20260810 00:03:13.800252 5558 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48654
I20260810 00:03:13.835294 5558 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:03:13.857229 5558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260810 00:03:13.903527 5780 heartbeater.cc:503] Master 127.5.100.126:44703 was elected leader, sending a full tablet report...
I20260810 00:03:13.919497 5743 tablet_service.cc:1467] Processing CreateTablet for tablet 91befdf118104d8da41391da96bb37df (DEFAULT_TABLE table=test-table [id=99ca9dfbaa7c4747b897f097788e5e59]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:13.921187 5743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91befdf118104d8da41391da96bb37df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:13.921149 5742 tablet_service.cc:1467] Processing CreateTablet for tablet 410157a5c77440fb9f78c343bc63b1fa (DEFAULT_TABLE table=test-table [id=99ca9dfbaa7c4747b897f097788e5e59]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:13.922600 5742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410157a5c77440fb9f78c343bc63b1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:13.924077 5660 tablet_service.cc:1467] Processing CreateTablet for tablet 91befdf118104d8da41391da96bb37df (DEFAULT_TABLE table=test-table [id=99ca9dfbaa7c4747b897f097788e5e59]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:13.925451 5660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91befdf118104d8da41391da96bb37df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:13.929347 5659 tablet_service.cc:1467] Processing CreateTablet for tablet 410157a5c77440fb9f78c343bc63b1fa (DEFAULT_TABLE table=test-table [id=99ca9dfbaa7c4747b897f097788e5e59]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:13.930634 5659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410157a5c77440fb9f78c343bc63b1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:13.941715 5797 tablet_bootstrap.cc:492] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: Bootstrap starting.
I20260810 00:03:13.948422 5797 tablet_bootstrap.cc:654] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:13.953799 5798 tablet_bootstrap.cc:492] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: Bootstrap starting.
I20260810 00:03:13.961512 5797 tablet_bootstrap.cc:492] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: No bootstrap required, opened a new log
I20260810 00:03:13.962008 5797 ts_tablet_manager.cc:1397] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.006s
I20260810 00:03:13.963407 5798 tablet_bootstrap.cc:654] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:13.964990 5797 raft_consensus.cc:348] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:13.965720 5797 raft_consensus.cc:729] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adc7b883bbd4f8ea378411cddf76869, State: Initialized, Role: FOLLOWER
I20260810 00:03:13.966828 5797 consensus_queue.cc:260] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:13.973839 5797 ts_tablet_manager.cc:1428] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260810 00:03:13.974517 5798 tablet_bootstrap.cc:492] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: No bootstrap required, opened a new log
I20260810 00:03:13.974931 5798 ts_tablet_manager.cc:1397] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.004s
I20260810 00:03:13.975028 5797 tablet_bootstrap.cc:492] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: Bootstrap starting.
I20260810 00:03:13.977571 5798 raft_consensus.cc:348] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:13.978271 5798 raft_consensus.cc:729] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Initialized, Role: FOLLOWER
I20260810 00:03:13.978833 5798 consensus_queue.cc:260] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:13.981518 5797 tablet_bootstrap.cc:654] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:13.992753 5798 ts_tablet_manager.cc:1428] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: Time spent starting tablet: real 0.018s user 0.006s sys 0.005s
I20260810 00:03:13.993381 5797 tablet_bootstrap.cc:492] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: No bootstrap required, opened a new log
I20260810 00:03:13.993934 5797 ts_tablet_manager.cc:1397] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20260810 00:03:13.995404 5798 tablet_bootstrap.cc:492] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: Bootstrap starting.
I20260810 00:03:13.996661 5797 raft_consensus.cc:348] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:13.997377 5797 raft_consensus.cc:729] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adc7b883bbd4f8ea378411cddf76869, State: Initialized, Role: FOLLOWER
I20260810 00:03:13.997943 5797 consensus_queue.cc:260] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:14.000209 5797 ts_tablet_manager.cc:1428] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 00:03:14.002112 5798 tablet_bootstrap.cc:654] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:14.012240 5798 tablet_bootstrap.cc:492] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: No bootstrap required, opened a new log
I20260810 00:03:14.012718 5798 ts_tablet_manager.cc:1397] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260810 00:03:14.015395 5798 raft_consensus.cc:348] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:14.016073 5798 raft_consensus.cc:729] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Initialized, Role: FOLLOWER
I20260810 00:03:14.016652 5798 consensus_queue.cc:260] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:14.018643 5798 ts_tablet_manager.cc:1428] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
W20260810 00:03:14.032306 5781 tablet.cc:2369] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:03:14.051535 5521 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260810 00:03:14.051884 5521 raft_consensus.cc:412] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Emulating election...
I20260810 00:03:14.052109 5521 raft_consensus.cc:3037] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:14.055949 5521 raft_consensus.cc:686] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Becoming Leader. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:14.056633 5521 consensus_queue.cc:237] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:14.060271 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:14.066704 5557 catalog_manager.cc:5495] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5af311658ce4377a6d60edbba6ca855 (127.5.100.66). New cstate: current_term: 1 leader_uuid: "f5af311658ce4377a6d60edbba6ca855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260810 00:03:14.089845 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:14.193701 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:14.397761 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:14.664446 5671 raft_consensus.cc:3037] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:14.672587 5809 consensus_queue.cc:1035] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:14.701509 5521 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260810 00:03:14.701797 5521 raft_consensus.cc:412] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Emulating election...
I20260810 00:03:14.702046 5521 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:14.707633 5521 raft_consensus.cc:686] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Becoming Leader. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:14.708287 5521 consensus_queue.cc:237] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:14.710734 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:14.714761 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:14.715477 5558 catalog_manager.cc:5495] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5af311658ce4377a6d60edbba6ca855 (127.5.100.66). New cstate: current_term: 1 leader_uuid: "f5af311658ce4377a6d60edbba6ca855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260810 00:03:14.818610 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:15.022614 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260810 00:03:15.301530 5671 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:15.307406 5814 consensus_queue.cc:1035] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:15.526176 5521 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260810 00:03:15.526465 5521 raft_consensus.cc:412] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Emulating election...
I20260810 00:03:15.526746 5521 raft_consensus.cc:3032] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 00:03:15.526996 5521 raft_consensus.cc:729] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:15.527565 5521 consensus_queue.cc:260] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:15.528584 5521 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:15.532097 5521 raft_consensus.cc:686] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 2 LEADER]: Becoming Leader. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:15.532617 5521 consensus_queue.cc:237] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:15.533931 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:15.536792 5557 catalog_manager.cc:5495] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "f5af311658ce4377a6d60edbba6ca855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260810 00:03:15.537263 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260810 00:03:15.640941 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260810 00:03:15.844321 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260810 00:03:16.082023 5671 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:16.086308 5671 raft_consensus.cc:1261] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 2 FOLLOWER]: Refusing update from remote peer f5af311658ce4377a6d60edbba6ca855: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260810 00:03:16.087272 5814 consensus_queue.cc:1035] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260810 00:03:16.147567 5521 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260810 00:03:16.147826 5521 raft_consensus.cc:412] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 2 FOLLOWER]: Emulating election...
I20260810 00:03:16.148069 5521 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:16.152030 5521 raft_consensus.cc:686] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 3 LEADER]: Becoming Leader. State: Replica: 7adc7b883bbd4f8ea378411cddf76869, State: Running, Role: LEADER
I20260810 00:03:16.152729 5521 consensus_queue.cc:237] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:16.154171 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:16.157716 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260810 00:03:16.157867 5557 catalog_manager.cc:5495] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 reported cstate change: term changed from 2 to 3, leader changed from f5af311658ce4377a6d60edbba6ca855 (127.5.100.66) to 7adc7b883bbd4f8ea378411cddf76869 (127.5.100.65). New cstate: current_term: 3 leader_uuid: "7adc7b883bbd4f8ea378411cddf76869" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:16.261113 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260810 00:03:16.465231 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260810 00:03:16.521071 5671 raft_consensus.cc:1226] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 3 LEADER]: Rejecting Update request from peer f5af311658ce4377a6d60edbba6ca855 for earlier term 2. Current term is 3. Ops: []
I20260810 00:03:16.522095 5833 consensus_queue.cc:1046] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 00:03:16.522738 5833 consensus_queue.cc:1225] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Peer 7adc7b883bbd4f8ea378411cddf76869 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260810 00:03:16.523164 5814 raft_consensus.cc:3032] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 2 LEADER]: Stepping down as leader of term 2
I20260810 00:03:16.523429 5814 raft_consensus.cc:729] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:16.523927 5814 consensus_queue.cc:260] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:16.524751 5814 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:16.619131 5753 raft_consensus.cc:1261] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 3 FOLLOWER]: Refusing update from remote peer 7adc7b883bbd4f8ea378411cddf76869: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260810 00:03:16.620281 5832 consensus_queue.cc:1035] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 00:03:16.968843 5521 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260810 00:03:16.969084 5521 raft_consensus.cc:412] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 1 FOLLOWER]: Emulating election...
I20260810 00:03:16.969256 5521 raft_consensus.cc:3037] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:16.973069 5521 raft_consensus.cc:686] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 2 LEADER]: Becoming Leader. State: Replica: 7adc7b883bbd4f8ea378411cddf76869, State: Running, Role: LEADER
I20260810 00:03:16.973712 5521 consensus_queue.cc:237] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:16.975486 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:16.979156 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260810 00:03:16.979372 5556 catalog_manager.cc:5495] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 reported cstate change: term changed from 1 to 2, leader changed from f5af311658ce4377a6d60edbba6ca855 (127.5.100.66) to 7adc7b883bbd4f8ea378411cddf76869 (127.5.100.65). New cstate: current_term: 2 leader_uuid: "7adc7b883bbd4f8ea378411cddf76869" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:17.082904 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260810 00:03:17.228896 5671 raft_consensus.cc:1226] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 2 LEADER]: Rejecting Update request from peer f5af311658ce4377a6d60edbba6ca855 for earlier term 1. Current term is 2. Ops: []
I20260810 00:03:17.230207 5851 consensus_queue.cc:1046] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260810 00:03:17.231196 5851 consensus_queue.cc:1225] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Peer 7adc7b883bbd4f8ea378411cddf76869 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260810 00:03:17.231583 5852 raft_consensus.cc:3032] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 00:03:17.231840 5852 raft_consensus.cc:729] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:17.232462 5852 consensus_queue.cc:260] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:17.233186 5852 raft_consensus.cc:3037] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:17.286461 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260810 00:03:17.366737 5753 raft_consensus.cc:1261] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 2 FOLLOWER]: Refusing update from remote peer 7adc7b883bbd4f8ea378411cddf76869: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260810 00:03:17.367882 5845 consensus_queue.cc:1035] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260810 00:03:17.589941 5521 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260810 00:03:17.590150 5521 raft_consensus.cc:412] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 3 FOLLOWER]: Emulating election...
I20260810 00:03:17.590332 5521 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 3 FOLLOWER]: Advancing to term 4
I20260810 00:03:17.594587 5521 raft_consensus.cc:686] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 4 LEADER]: Becoming Leader. State: Replica: f5af311658ce4377a6d60edbba6ca855, State: Running, Role: LEADER
I20260810 00:03:17.595261 5521 consensus_queue.cc:237] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:17.596565 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:17.599783 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260810 00:03:17.599903 5556 catalog_manager.cc:5495] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 reported cstate change: term changed from 3 to 4, leader changed from 7adc7b883bbd4f8ea378411cddf76869 (127.5.100.65) to f5af311658ce4377a6d60edbba6ca855 (127.5.100.66). New cstate: current_term: 4 leader_uuid: "f5af311658ce4377a6d60edbba6ca855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260810 00:03:17.703253 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260810 00:03:17.906988 5521 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260810 00:03:18.139423 5753 raft_consensus.cc:1226] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 4 LEADER]: Rejecting Update request from peer 7adc7b883bbd4f8ea378411cddf76869 for earlier term 3. Current term is 4. Ops: []
I20260810 00:03:18.140583 5845 consensus_queue.cc:1046] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260810 00:03:18.141810 5845 consensus_queue.cc:1225] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [LEADER]: Peer f5af311658ce4377a6d60edbba6ca855 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260810 00:03:18.142309 5862 raft_consensus.cc:3032] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 3 LEADER]: Stepping down as leader of term 3
I20260810 00:03:18.142551 5862 raft_consensus.cc:729] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7adc7b883bbd4f8ea378411cddf76869, State: Running, Role: LEADER
I20260810 00:03:18.143018 5862 consensus_queue.cc:260] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 } } peers { permanent_uuid: "f5af311658ce4377a6d60edbba6ca855" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39377 } }
I20260810 00:03:18.143747 5862 raft_consensus.cc:3037] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 3 FOLLOWER]: Advancing to term 4
I20260810 00:03:18.152365 5671 raft_consensus.cc:1261] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 4 FOLLOWER]: Refusing update from remote peer f5af311658ce4377a6d60edbba6ca855: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260810 00:03:18.153501 5863 consensus_queue.cc:1035] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adc7b883bbd4f8ea378411cddf76869" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 43477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260810 00:03:18.412748 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:18.432698 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:18.433293 5521 tablet_replica.cc:330] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869: stopping tablet replica
I20260810 00:03:18.433882 5521 raft_consensus.cc:2229] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:18.434255 5521 raft_consensus.cc:2258] T 410157a5c77440fb9f78c343bc63b1fa P 7adc7b883bbd4f8ea378411cddf76869 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:18.436386 5521 tablet_replica.cc:330] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869: stopping tablet replica
I20260810 00:03:18.436766 5521 raft_consensus.cc:2229] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 2 LEADER]: Raft consensus shutting down.
I20260810 00:03:18.437232 5521 raft_consensus.cc:2258] T 91befdf118104d8da41391da96bb37df P 7adc7b883bbd4f8ea378411cddf76869 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:18.543809 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:18.553273 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:03:18.569301 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:18.569839 5521 tablet_replica.cc:330] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855: stopping tablet replica
I20260810 00:03:18.570348 5521 raft_consensus.cc:2229] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:18.570724 5521 raft_consensus.cc:2258] T 91befdf118104d8da41391da96bb37df P f5af311658ce4377a6d60edbba6ca855 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:18.572029 5521 tablet_replica.cc:330] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855: stopping tablet replica
I20260810 00:03:18.572532 5521 raft_consensus.cc:2229] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 4 LEADER]: Raft consensus shutting down.
I20260810 00:03:18.573058 5521 raft_consensus.cc:2258] T 410157a5c77440fb9f78c343bc63b1fa P f5af311658ce4377a6d60edbba6ca855 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:18.591163 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:18.598976 5521 master.cc:537] Master@127.5.100.126:44703 shutting down...
I20260810 00:03:18.755106 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:18.755592 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:18.755882 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7d0a134d5c8744f0be216b16f2551e98: stopping tablet replica
I20260810 00:03:18.771796 5521 master.cc:552] Master@127.5.100.126:44703 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7106 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260810 00:03:18.797335 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:40825
I20260810 00:03:18.798333 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:18.802676 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:18.803768 5873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:18.805369 5875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:18.806277 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:18.806864 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:18.807050 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:18.807201 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320198807187 us; error 0 us; skew 500 ppm
I20260810 00:03:18.807685 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:18.813386 5521 webserver.cc:460] Webserver started at http://127.5.100.126:46865/ using document root <none> and password file <none>
I20260810 00:03:18.813897 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:18.814105 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:18.814349 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:18.815361 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "a28cebdffc4949d3a429cf3fb786ac45"
format_stamp: "Formatted at 2026-08-10 00:03:18 on dist-test-slave-8w9v"
I20260810 00:03:18.819269 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260810 00:03:18.822207 5880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:18.823006 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:03:18.823249 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "a28cebdffc4949d3a429cf3fb786ac45"
format_stamp: "Formatted at 2026-08-10 00:03:18 on dist-test-slave-8w9v"
I20260810 00:03:18.823498 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:18.833999 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:18.834849 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:18.866689 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:40825
I20260810 00:03:18.866784 5934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:40825 every 8 connection(s)
I20260810 00:03:18.870378 5935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:18.878582 5935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45: Bootstrap starting.
I20260810 00:03:18.882156 5935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:18.885488 5935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45: No bootstrap required, opened a new log
I20260810 00:03:18.886927 5935 raft_consensus.cc:348] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER }
I20260810 00:03:18.887238 5935 raft_consensus.cc:374] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:18.887413 5935 raft_consensus.cc:729] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a28cebdffc4949d3a429cf3fb786ac45, State: Initialized, Role: FOLLOWER
I20260810 00:03:18.887918 5935 consensus_queue.cc:260] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER }
I20260810 00:03:18.888379 5935 raft_consensus.cc:388] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:18.888543 5935 raft_consensus.cc:482] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:18.888723 5935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:18.891800 5935 raft_consensus.cc:504] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER }
I20260810 00:03:18.892275 5935 leader_election.cc:302] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a28cebdffc4949d3a429cf3fb786ac45; no voters:
I20260810 00:03:18.893260 5935 leader_election.cc:288] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:18.893554 5938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:18.894953 5938 raft_consensus.cc:686] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 1 LEADER]: Becoming Leader. State: Replica: a28cebdffc4949d3a429cf3fb786ac45, State: Running, Role: LEADER
I20260810 00:03:18.895481 5938 consensus_queue.cc:237] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER }
I20260810 00:03:18.895884 5935 sys_catalog.cc:564] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:18.898524 5940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a28cebdffc4949d3a429cf3fb786ac45. Latest consensus state: current_term: 1 leader_uuid: "a28cebdffc4949d3a429cf3fb786ac45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER } }
I20260810 00:03:18.898463 5939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a28cebdffc4949d3a429cf3fb786ac45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28cebdffc4949d3a429cf3fb786ac45" member_type: VOTER } }
I20260810 00:03:18.899289 5939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:18.899893 5940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:18.901245 5944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:18.908980 5944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:18.909897 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:18.916669 5944 catalog_manager.cc:1347] Generated new cluster ID: 8248e8f1e9784daebc7dd419f3aa8aa4
I20260810 00:03:18.916936 5944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:18.933876 5944 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:18.935313 5944 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:18.955924 5944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45: Generated new TSK 0
I20260810 00:03:18.956566 5944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:18.975682 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:18.980980 5956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:18.982267 5957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:18.983484 5959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:18.984203 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:18.985038 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:18.985196 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:18.985397 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320198985366 us; error 0 us; skew 500 ppm
I20260810 00:03:18.985890 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:18.987982 5521 webserver.cc:460] Webserver started at http://127.5.100.65:32779/ using document root <none> and password file <none>
I20260810 00:03:18.988466 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:18.988648 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:18.988912 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:18.989907 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "b65ee2d6b4e24fe4a0c81be8f2c78470"
format_stamp: "Formatted at 2026-08-10 00:03:18 on dist-test-slave-8w9v"
I20260810 00:03:18.994005 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 00:03:18.998466 5964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:18.999516 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:03:18.999851 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "b65ee2d6b4e24fe4a0c81be8f2c78470"
format_stamp: "Formatted at 2026-08-10 00:03:18 on dist-test-slave-8w9v"
I20260810 00:03:19.000244 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:19.013314 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:19.014343 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:19.015655 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:19.017772 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:19.017990 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.018232 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:19.018395 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.053579 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:41229
I20260810 00:03:19.053673 6032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:41229 every 8 connection(s)
I20260810 00:03:19.057921 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:19.065310 6038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.066392 6039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.069195 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:19.070263 6043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.071338 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:19.071624 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:19.071797 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320199071781 us; error 0 us; skew 500 ppm
I20260810 00:03:19.072513 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:19.072808 6033 heartbeater.cc:347] Connected to a master server at 127.5.100.126:40825
I20260810 00:03:19.073249 6033 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:19.074139 6033 heartbeater.cc:511] Master 127.5.100.126:40825 requested a full tablet report, sending...
I20260810 00:03:19.075376 5521 webserver.cc:460] Webserver started at http://127.5.100.66:36925/ using document root <none> and password file <none>
I20260810 00:03:19.075970 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:19.076195 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:19.076506 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:19.076491 5897 ts_manager.cc:194] Registered new tserver with Master: b65ee2d6b4e24fe4a0c81be8f2c78470 (127.5.100.65:41229)
I20260810 00:03:19.077625 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "e372e0bc8f034f769790ebbf8cf074ed"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.078202 5897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55050
I20260810 00:03:19.082624 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 00:03:19.085670 6047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.086331 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:03:19.086592 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "e372e0bc8f034f769790ebbf8cf074ed"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.086869 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:19.119728 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:19.120783 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:19.122165 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:19.124558 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:19.124789 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.125037 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:19.125233 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.159634 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:37215
I20260810 00:03:19.159713 6119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:37215 every 8 connection(s)
I20260810 00:03:19.174163 6120 heartbeater.cc:347] Connected to a master server at 127.5.100.126:40825
I20260810 00:03:19.174491 6120 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:19.175094 6120 heartbeater.cc:511] Master 127.5.100.126:40825 requested a full tablet report, sending...
I20260810 00:03:19.176772 5897 ts_manager.cc:194] Registered new tserver with Master: e372e0bc8f034f769790ebbf8cf074ed (127.5.100.66:37215)
I20260810 00:03:19.177662 5521 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015222394s
I20260810 00:03:19.178220 5897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55060
I20260810 00:03:19.181262 6033 heartbeater.cc:503] Master 127.5.100.126:40825 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260810 00:03:19.199016 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:19.214030 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:19.229140 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:19.235344 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:03:19.248456 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:19.263484 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:19.269590 5521 master.cc:537] Master@127.5.100.126:40825 shutting down...
I20260810 00:03:19.282785 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:19.283248 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:19.283607 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P a28cebdffc4949d3a429cf3fb786ac45: stopping tablet replica
I20260810 00:03:19.301121 5521 master.cc:552] Master@127.5.100.126:40825 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (523 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260810 00:03:19.320001 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:33695
I20260810 00:03:19.320919 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:19.325227 6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.326182 6135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.327482 6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.328059 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:19.328797 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:19.328955 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:19.329134 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320199329107 us; error 0 us; skew 500 ppm
I20260810 00:03:19.329608 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:19.331727 5521 webserver.cc:460] Webserver started at http://127.5.100.126:41285/ using document root <none> and password file <none>
I20260810 00:03:19.332285 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:19.332469 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:19.332723 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:19.333717 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "983e0346b6cb4a379c383426dc729592"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.337787 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:03:19.340816 6142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.341496 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:03:19.341758 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "983e0346b6cb4a379c383426dc729592"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.342010 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:19.357784 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:19.358655 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:19.390838 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:33695
I20260810 00:03:19.390961 6200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:33695 every 8 connection(s)
I20260810 00:03:19.394188 6201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:19.402402 6201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592: Bootstrap starting.
I20260810 00:03:19.405876 6201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:19.409232 6201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592: No bootstrap required, opened a new log
I20260810 00:03:19.410822 6201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER }
I20260810 00:03:19.411193 6201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:19.411373 6201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 983e0346b6cb4a379c383426dc729592, State: Initialized, Role: FOLLOWER
I20260810 00:03:19.411805 6201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER }
I20260810 00:03:19.412307 6201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:19.412506 6201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:19.412704 6201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:19.416592 6201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER }
I20260810 00:03:19.417186 6201 leader_election.cc:302] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 983e0346b6cb4a379c383426dc729592; no voters:
I20260810 00:03:19.418464 6201 leader_election.cc:288] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:19.418798 6204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:19.420189 6204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 1 LEADER]: Becoming Leader. State: Replica: 983e0346b6cb4a379c383426dc729592, State: Running, Role: LEADER
I20260810 00:03:19.420848 6204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER }
I20260810 00:03:19.421258 6201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:19.423588 6205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "983e0346b6cb4a379c383426dc729592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER } }
I20260810 00:03:19.423622 6206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 983e0346b6cb4a379c383426dc729592. Latest consensus state: current_term: 1 leader_uuid: "983e0346b6cb4a379c383426dc729592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983e0346b6cb4a379c383426dc729592" member_type: VOTER } }
I20260810 00:03:19.424228 6205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:19.424382 6206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:19.426882 6209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:19.431427 6209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:19.436229 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:19.439390 6209 catalog_manager.cc:1347] Generated new cluster ID: 5a4527935b954a75a46595764457da69
I20260810 00:03:19.439643 6209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:19.458169 6209 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:19.459555 6209 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:19.476096 6209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592: Generated new TSK 0
I20260810 00:03:19.476687 6209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:19.502442 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:19.507826 6222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.508867 6223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.510138 6225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.510653 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:19.511353 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:19.511556 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:19.511708 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320199511691 us; error 0 us; skew 500 ppm
I20260810 00:03:19.512270 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:19.514312 5521 webserver.cc:460] Webserver started at http://127.5.100.65:41545/ using document root <none> and password file <none>
I20260810 00:03:19.514739 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:19.514931 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:19.515192 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:19.516275 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "b3b572adea164ea188a15f8a9abe90ef"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.520612 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:03:19.523670 6230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.524471 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:03:19.524734 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "b3b572adea164ea188a15f8a9abe90ef"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.525004 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:19.559011 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:19.559945 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:19.561223 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:19.563197 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:19.563387 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.563630 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:19.563804 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.597811 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:32927
I20260810 00:03:19.597896 6297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:32927 every 8 connection(s)
I20260810 00:03:19.601783 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:19.608521 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:19.610069 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.612789 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:19.613262 6305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:19.614184 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:19.614440 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:19.614622 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320199614606 us; error 0 us; skew 500 ppm
I20260810 00:03:19.615336 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:19.616201 6298 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33695
I20260810 00:03:19.616518 6298 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:19.617203 6298 heartbeater.cc:511] Master 127.5.100.126:33695 requested a full tablet report, sending...
I20260810 00:03:19.618213 5521 webserver.cc:460] Webserver started at http://127.5.100.66:32823/ using document root <none> and password file <none>
I20260810 00:03:19.618758 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:19.618942 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:19.619056 6161 ts_manager.cc:194] Registered new tserver with Master: b3b572adea164ea188a15f8a9abe90ef (127.5.100.65:32927)
I20260810 00:03:19.619239 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:19.620360 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "21a93b98bd2b4fd5a9fd740532e24d37"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.620827 6161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55790
I20260810 00:03:19.624696 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:03:19.627602 6310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.628294 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:03:19.628539 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "21a93b98bd2b4fd5a9fd740532e24d37"
format_stamp: "Formatted at 2026-08-10 00:03:19 on dist-test-slave-8w9v"
I20260810 00:03:19.628808 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:19.643314 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:19.644201 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:19.645448 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:19.647433 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:19.647639 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.647858 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:19.648013 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:19.681136 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:40411
I20260810 00:03:19.681214 6376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:40411 every 8 connection(s)
I20260810 00:03:19.695026 6377 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33695
I20260810 00:03:19.695389 6377 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:19.695951 6377 heartbeater.cc:511] Master 127.5.100.126:33695 requested a full tablet report, sending...
I20260810 00:03:19.697669 6161 ts_manager.cc:194] Registered new tserver with Master: 21a93b98bd2b4fd5a9fd740532e24d37 (127.5.100.66:40411)
I20260810 00:03:19.698640 5521 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.014429295s
I20260810 00:03:19.699036 6161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55792
I20260810 00:03:19.718488 6161 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 00:03:19.719720 6161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260810 00:03:19.723400 6298 heartbeater.cc:503] Master 127.5.100.126:33695 was elected leader, sending a full tablet report...
I20260810 00:03:19.751675 6341 tablet_service.cc:1467] Processing CreateTablet for tablet 26c96105b787422c80eb9f5aa119989c (DEFAULT_TABLE table=test-table [id=0cb57a0bd90f4821b490e1b29603f96a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:19.752820 6341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c96105b787422c80eb9f5aa119989c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:19.759773 6261 tablet_service.cc:1467] Processing CreateTablet for tablet 26c96105b787422c80eb9f5aa119989c (DEFAULT_TABLE table=test-table [id=0cb57a0bd90f4821b490e1b29603f96a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:19.760637 6261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c96105b787422c80eb9f5aa119989c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:19.763229 6391 tablet_bootstrap.cc:492] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: Bootstrap starting.
I20260810 00:03:19.767903 6391 tablet_bootstrap.cc:654] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:19.771343 6392 tablet_bootstrap.cc:492] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: Bootstrap starting.
I20260810 00:03:19.772171 6391 tablet_bootstrap.cc:492] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: No bootstrap required, opened a new log
I20260810 00:03:19.772639 6391 ts_tablet_manager.cc:1397] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 00:03:19.774269 6391 raft_consensus.cc:348] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:19.774739 6391 raft_consensus.cc:729] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a93b98bd2b4fd5a9fd740532e24d37, State: Initialized, Role: FOLLOWER
I20260810 00:03:19.775156 6391 consensus_queue.cc:260] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:19.775552 6392 tablet_bootstrap.cc:654] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:19.777302 6377 heartbeater.cc:503] Master 127.5.100.126:33695 was elected leader, sending a full tablet report...
I20260810 00:03:19.777503 6391 ts_tablet_manager.cc:1428] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260810 00:03:19.779742 6392 tablet_bootstrap.cc:492] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: No bootstrap required, opened a new log
I20260810 00:03:19.780093 6392 ts_tablet_manager.cc:1397] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:03:19.781818 6392 raft_consensus.cc:348] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:19.782299 6392 raft_consensus.cc:729] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Initialized, Role: FOLLOWER
I20260810 00:03:19.782748 6392 consensus_queue.cc:260] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:19.785770 6392 ts_tablet_manager.cc:1428] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260810 00:03:19.807165 5521 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260810 00:03:19.807348 5521 raft_consensus.cc:412] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 0 FOLLOWER]: Emulating election...
I20260810 00:03:19.807504 5521 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:19.811192 5521 raft_consensus.cc:686] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 1 LEADER]: Becoming Leader. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Running, Role: LEADER
W20260810 00:03:19.811519 6298 heartbeater.cc:644] Failed to heartbeat to 127.5.100.126:33695 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260810 00:03:19.811714 5521 consensus_queue.cc:237] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:19.814196 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:19.834406 5521 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260810 00:03:19.938251 5521 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260810 00:03:20.142148 5521 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260810 00:03:20.439383 6352 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:20.445101 6399 consensus_queue.cc:1035] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:20.445693 5521 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260810 00:03:20.850756 5521 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260810 00:03:20.850975 5521 raft_consensus.cc:412] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 1 LEADER]: Emulating election...
I20260810 00:03:20.851229 5521 raft_consensus.cc:3032] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 1 LEADER]: Stepping down as leader of term 1
I20260810 00:03:20.851462 5521 raft_consensus.cc:729] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Running, Role: LEADER
I20260810 00:03:20.851945 5521 consensus_queue.cc:260] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:20.852634 5521 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:20.856571 5521 raft_consensus.cc:686] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 2 LEADER]: Becoming Leader. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Running, Role: LEADER
I20260810 00:03:20.857154 5521 consensus_queue.cc:237] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:20.858450 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:20.861698 5521 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260810 00:03:20.965257 5521 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260810 00:03:21.169095 5521 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260810 00:03:21.304850 6352 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:21.309212 6352 raft_consensus.cc:1261] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 2 FOLLOWER]: Refusing update from remote peer b3b572adea164ea188a15f8a9abe90ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260810 00:03:21.310349 6399 consensus_queue.cc:1035] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260810 00:03:21.473758 5521 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260810 00:03:21.474050 5521 raft_consensus.cc:412] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 2 LEADER]: Emulating election...
I20260810 00:03:21.474258 5521 raft_consensus.cc:3032] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 2 LEADER]: Stepping down as leader of term 2
I20260810 00:03:21.474455 5521 raft_consensus.cc:729] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Running, Role: LEADER
I20260810 00:03:21.474884 5521 consensus_queue.cc:260] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:21.475507 5521 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:21.479385 5521 raft_consensus.cc:686] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 3 LEADER]: Becoming Leader. State: Replica: b3b572adea164ea188a15f8a9abe90ef, State: Running, Role: LEADER
I20260810 00:03:21.479852 5521 consensus_queue.cc:237] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 } } peers { permanent_uuid: "b3b572adea164ea188a15f8a9abe90ef" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 32927 } }
I20260810 00:03:21.481108 5521 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260810 00:03:21.484444 5521 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260810 00:03:21.588145 5521 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260810 00:03:21.793372 5521 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260810 00:03:22.039744 6352 raft_consensus.cc:3037] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:22.046103 6352 raft_consensus.cc:1261] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 3 FOLLOWER]: Refusing update from remote peer b3b572adea164ea188a15f8a9abe90ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260810 00:03:22.047472 6415 consensus_queue.cc:1035] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a93b98bd2b4fd5a9fd740532e24d37" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260810 00:03:22.102339 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:22.126070 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:22.126755 5521 tablet_replica.cc:330] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef: stopping tablet replica
I20260810 00:03:22.127305 5521 raft_consensus.cc:2229] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 3 LEADER]: Raft consensus shutting down.
I20260810 00:03:22.127871 5521 raft_consensus.cc:2258] T 26c96105b787422c80eb9f5aa119989c P b3b572adea164ea188a15f8a9abe90ef [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:22.146804 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:22.155252 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:03:22.171610 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:22.172192 5521 tablet_replica.cc:330] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37: stopping tablet replica
I20260810 00:03:22.172693 5521 raft_consensus.cc:2229] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:22.173089 5521 raft_consensus.cc:2258] T 26c96105b787422c80eb9f5aa119989c P 21a93b98bd2b4fd5a9fd740532e24d37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:22.191022 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:22.198947 5521 master.cc:537] Master@127.5.100.126:33695 shutting down...
I20260810 00:03:22.216504 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:22.217054 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:22.217379 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 983e0346b6cb4a379c383426dc729592: stopping tablet replica
I20260810 00:03:22.233915 5521 master.cc:552] Master@127.5.100.126:33695 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2942 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10573 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260810 00:03:22.264156 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:46527
I20260810 00:03:22.265286 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:22.270907 6425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.271811 6426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.273213 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:22.273833 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:22.274435 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:22.274597 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:22.274715 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320202274708 us; error 0 us; skew 500 ppm
I20260810 00:03:22.275132 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:22.277429 5521 webserver.cc:460] Webserver started at http://127.5.100.126:38097/ using document root <none> and password file <none>
I20260810 00:03:22.277830 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:22.277969 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:22.278183 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:22.279225 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "607c1e6f945c4729874cd83b97790323"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.283252 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 00:03:22.286259 6435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.286939 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:03:22.287199 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "607c1e6f945c4729874cd83b97790323"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.287482 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:22.308846 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:22.310065 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:22.346879 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:46527
I20260810 00:03:22.346974 6496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:46527 every 8 connection(s)
I20260810 00:03:22.350473 6497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:22.359047 6497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323: Bootstrap starting.
I20260810 00:03:22.363420 6497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:22.367252 6497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323: No bootstrap required, opened a new log
I20260810 00:03:22.369122 6497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER }
I20260810 00:03:22.369541 6497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:22.369792 6497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607c1e6f945c4729874cd83b97790323, State: Initialized, Role: FOLLOWER
I20260810 00:03:22.370370 6497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER }
I20260810 00:03:22.370806 6497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:22.371028 6497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:22.371325 6497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:22.375145 6497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER }
I20260810 00:03:22.375636 6497 leader_election.cc:302] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607c1e6f945c4729874cd83b97790323; no voters:
I20260810 00:03:22.376793 6497 leader_election.cc:288] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:22.377169 6501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:22.378470 6501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 1 LEADER]: Becoming Leader. State: Replica: 607c1e6f945c4729874cd83b97790323, State: Running, Role: LEADER
I20260810 00:03:22.379029 6501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER }
I20260810 00:03:22.379665 6497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:22.381780 6503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607c1e6f945c4729874cd83b97790323. Latest consensus state: current_term: 1 leader_uuid: "607c1e6f945c4729874cd83b97790323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER } }
I20260810 00:03:22.381862 6502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "607c1e6f945c4729874cd83b97790323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607c1e6f945c4729874cd83b97790323" member_type: VOTER } }
I20260810 00:03:22.382344 6503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:22.382478 6502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:22.385526 6507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:22.393126 6507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:22.394137 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:22.401417 6507 catalog_manager.cc:1347] Generated new cluster ID: 6f89b156e30646a58dedb6d5d927c65c
I20260810 00:03:22.401655 6507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:22.441839 6507 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:22.443344 6507 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:22.455977 6507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323: Generated new TSK 0
I20260810 00:03:22.456537 6507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:22.460578 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:22.466060 6522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.467272 6523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.468652 6525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:22.469053 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:22.469825 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:22.470008 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:22.470161 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320202470133 us; error 0 us; skew 500 ppm
I20260810 00:03:22.470671 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:22.472906 5521 webserver.cc:460] Webserver started at http://127.5.100.65:38559/ using document root <none> and password file <none>
I20260810 00:03:22.473359 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:22.473553 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:22.473822 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:22.474887 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "4d591fe156374503b96092336507b7b4"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.479115 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:03:22.482313 6530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.483000 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:03:22.483278 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "4d591fe156374503b96092336507b7b4"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.483567 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:22.510190 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:22.511252 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:22.512682 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:22.514756 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:22.514973 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.515204 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:22.515426 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.551851 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:42357
I20260810 00:03:22.551975 6600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:42357 every 8 connection(s)
I20260810 00:03:22.556253 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:22.564738 6605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.566036 6606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.567396 6608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:22.568022 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:22.568826 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:22.569016 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:22.569192 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320202569178 us; error 0 us; skew 500 ppm
I20260810 00:03:22.569741 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:22.571014 6601 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46527
I20260810 00:03:22.571345 6601 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:22.571883 6601 heartbeater.cc:511] Master 127.5.100.126:46527 requested a full tablet report, sending...
I20260810 00:03:22.572348 5521 webserver.cc:460] Webserver started at http://127.5.100.66:46441/ using document root <none> and password file <none>
I20260810 00:03:22.572914 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:22.573170 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:22.573447 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:22.573939 6454 ts_manager.cc:194] Registered new tserver with Master: 4d591fe156374503b96092336507b7b4 (127.5.100.65:42357)
I20260810 00:03:22.574625 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "38d5cf32084a48108771b955232a317d"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.575682 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33882
I20260810 00:03:22.578933 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:03:22.581898 6614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.582532 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:03:22.582821 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "38d5cf32084a48108771b955232a317d"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.583084 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:22.602705 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:22.603847 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:22.605154 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:22.607182 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:22.607375 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.607600 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:22.607772 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.644892 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:36909
I20260810 00:03:22.644989 6680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:36909 every 8 connection(s)
I20260810 00:03:22.649488 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:22.655889 6685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:22.658351 6686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:22.659566 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:22.661566 6688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:22.662556 6681 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46527
I20260810 00:03:22.662598 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:22.662889 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:22.662933 6681 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:22.663077 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320202663050 us; error 0 us; skew 500 ppm
I20260810 00:03:22.663676 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:22.663681 6681 heartbeater.cc:511] Master 127.5.100.126:46527 requested a full tablet report, sending...
I20260810 00:03:22.665949 6454 ts_manager.cc:194] Registered new tserver with Master: 38d5cf32084a48108771b955232a317d (127.5.100.66:36909)
I20260810 00:03:22.666258 5521 webserver.cc:460] Webserver started at http://127.5.100.67:44681/ using document root <none> and password file <none>
I20260810 00:03:22.666821 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:22.667052 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:22.667369 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:22.668025 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33898
I20260810 00:03:22.668859 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-2-root/instance:
uuid: "9cc268cc6df8461c8585fb1f8817c480"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.673031 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:03:22.676229 6693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.676904 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 00:03:22.677171 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-2-root
uuid: "9cc268cc6df8461c8585fb1f8817c480"
format_stamp: "Formatted at 2026-08-10 00:03:22 on dist-test-slave-8w9v"
I20260810 00:03:22.677446 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786320191480876-5521-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:22.696213 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:22.697410 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:22.698690 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:22.700770 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:22.700986 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.701210 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:22.701373 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:22.768447 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.67:35573
I20260810 00:03:22.768520 6760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.67:35573 every 8 connection(s)
I20260810 00:03:22.785735 6761 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46527
I20260810 00:03:22.786111 6761 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:22.786696 6761 heartbeater.cc:511] Master 127.5.100.126:46527 requested a full tablet report, sending...
I20260810 00:03:22.788614 6454 ts_manager.cc:194] Registered new tserver with Master: 9cc268cc6df8461c8585fb1f8817c480 (127.5.100.67:35573)
I20260810 00:03:22.789490 5521 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016030959s
I20260810 00:03:22.790447 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33900
I20260810 00:03:22.814505 6454 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260810 00:03:22.815984 6454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:03:22.856837 6644 tablet_service.cc:1467] Processing CreateTablet for tablet c1df93c7ecfa4c92ac07b5a31fcad69c (DEFAULT_TABLE table=test-table [id=c026182d5d43410194e66626964a1a1e]), partition=
I20260810 00:03:22.858028 6644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1df93c7ecfa4c92ac07b5a31fcad69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:22.861195 6722 tablet_service.cc:1467] Processing CreateTablet for tablet c1df93c7ecfa4c92ac07b5a31fcad69c (DEFAULT_TABLE table=test-table [id=c026182d5d43410194e66626964a1a1e]), partition=
I20260810 00:03:22.862211 6722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1df93c7ecfa4c92ac07b5a31fcad69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:22.866255 6562 tablet_service.cc:1467] Processing CreateTablet for tablet c1df93c7ecfa4c92ac07b5a31fcad69c (DEFAULT_TABLE table=test-table [id=c026182d5d43410194e66626964a1a1e]), partition=
I20260810 00:03:22.867197 6562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1df93c7ecfa4c92ac07b5a31fcad69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:22.870468 6776 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: Bootstrap starting.
I20260810 00:03:22.874835 6776 tablet_bootstrap.cc:654] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:22.877450 6777 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: Bootstrap starting.
I20260810 00:03:22.881372 6776 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: No bootstrap required, opened a new log
I20260810 00:03:22.881763 6776 ts_tablet_manager.cc:1397] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260810 00:03:22.884990 6777 tablet_bootstrap.cc:654] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:22.885069 6776 raft_consensus.cc:348] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.886022 6776 raft_consensus.cc:374] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:22.886371 6776 raft_consensus.cc:729] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d5cf32084a48108771b955232a317d, State: Initialized, Role: FOLLOWER
I20260810 00:03:22.887207 6776 consensus_queue.cc:260] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.889823 6777 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: No bootstrap required, opened a new log
I20260810 00:03:22.889822 6681 heartbeater.cc:503] Master 127.5.100.126:46527 was elected leader, sending a full tablet report...
I20260810 00:03:22.890138 6779 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: Bootstrap starting.
I20260810 00:03:22.891291 6776 ts_tablet_manager.cc:1428] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260810 00:03:22.891330 6777 ts_tablet_manager.cc:1397] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260810 00:03:22.894951 6777 raft_consensus.cc:348] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.895676 6777 raft_consensus.cc:374] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:22.895973 6777 raft_consensus.cc:729] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc268cc6df8461c8585fb1f8817c480, State: Initialized, Role: FOLLOWER
I20260810 00:03:22.896646 6777 consensus_queue.cc:260] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.897831 6779 tablet_bootstrap.cc:654] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:22.898720 6761 heartbeater.cc:503] Master 127.5.100.126:46527 was elected leader, sending a full tablet report...
I20260810 00:03:22.899381 6777 ts_tablet_manager.cc:1428] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 00:03:22.904583 6779 tablet_bootstrap.cc:492] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: No bootstrap required, opened a new log
I20260810 00:03:22.905061 6779 ts_tablet_manager.cc:1397] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.001s
I20260810 00:03:22.907475 6779 raft_consensus.cc:348] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.908113 6779 raft_consensus.cc:374] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:22.908391 6779 raft_consensus.cc:729] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d591fe156374503b96092336507b7b4, State: Initialized, Role: FOLLOWER
I20260810 00:03:22.908906 6779 consensus_queue.cc:260] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.911703 6601 heartbeater.cc:503] Master 127.5.100.126:46527 was elected leader, sending a full tablet report...
I20260810 00:03:22.912650 6779 ts_tablet_manager.cc:1428] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260810 00:03:22.932359 6781 raft_consensus.cc:482] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:22.932803 6781 raft_consensus.cc:504] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.934837 6781 leader_election.cc:288] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cc268cc6df8461c8585fb1f8817c480 (127.5.100.67:35573), 4d591fe156374503b96092336507b7b4 (127.5.100.65:42357)
I20260810 00:03:22.947168 6733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1df93c7ecfa4c92ac07b5a31fcad69c" candidate_uuid: "38d5cf32084a48108771b955232a317d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cc268cc6df8461c8585fb1f8817c480" is_pre_election: true
I20260810 00:03:22.947921 6733 raft_consensus.cc:2454] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38d5cf32084a48108771b955232a317d in term 0.
I20260810 00:03:22.949216 6616 leader_election.cc:302] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d5cf32084a48108771b955232a317d, 9cc268cc6df8461c8585fb1f8817c480; no voters:
I20260810 00:03:22.949787 6573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1df93c7ecfa4c92ac07b5a31fcad69c" candidate_uuid: "38d5cf32084a48108771b955232a317d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d591fe156374503b96092336507b7b4" is_pre_election: true
I20260810 00:03:22.950076 6781 raft_consensus.cc:2781] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:22.950418 6781 raft_consensus.cc:482] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:22.950492 6573 raft_consensus.cc:2454] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38d5cf32084a48108771b955232a317d in term 0.
I20260810 00:03:22.950793 6781 raft_consensus.cc:3037] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:22.955437 6781 raft_consensus.cc:504] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.957134 6781 leader_election.cc:288] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [CANDIDATE]: Term 1 election: Requested vote from peers 9cc268cc6df8461c8585fb1f8817c480 (127.5.100.67:35573), 4d591fe156374503b96092336507b7b4 (127.5.100.65:42357)
I20260810 00:03:22.957857 6733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1df93c7ecfa4c92ac07b5a31fcad69c" candidate_uuid: "38d5cf32084a48108771b955232a317d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cc268cc6df8461c8585fb1f8817c480"
I20260810 00:03:22.958114 6573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1df93c7ecfa4c92ac07b5a31fcad69c" candidate_uuid: "38d5cf32084a48108771b955232a317d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d591fe156374503b96092336507b7b4"
I20260810 00:03:22.958451 6733 raft_consensus.cc:3037] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:22.958669 6573 raft_consensus.cc:3037] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:22.963505 6573 raft_consensus.cc:2454] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38d5cf32084a48108771b955232a317d in term 1.
I20260810 00:03:22.963634 6733 raft_consensus.cc:2454] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38d5cf32084a48108771b955232a317d in term 1.
I20260810 00:03:22.964468 6617 leader_election.cc:302] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d5cf32084a48108771b955232a317d, 4d591fe156374503b96092336507b7b4; no voters:
I20260810 00:03:22.965174 6781 raft_consensus.cc:2781] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:22.966008 6781 raft_consensus.cc:686] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 1 LEADER]: Becoming Leader. State: Replica: 38d5cf32084a48108771b955232a317d, State: Running, Role: LEADER
I20260810 00:03:22.967046 6781 consensus_queue.cc:237] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } }
I20260810 00:03:22.973862 6453 catalog_manager.cc:5495] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d reported cstate change: term changed from 0 to 1, leader changed from <none> to 38d5cf32084a48108771b955232a317d (127.5.100.66). New cstate: current_term: 1 leader_uuid: "38d5cf32084a48108771b955232a317d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d5cf32084a48108771b955232a317d" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 36909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:23.371452 6781 consensus_queue.cc:1035] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cc268cc6df8461c8585fb1f8817c480" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 35573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:23.381072 6787 consensus_queue.cc:1035] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d591fe156374503b96092336507b7b4" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:24.032723 5521 tablet_replica.cc:330] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4: stopping tablet replica
I20260810 00:03:24.033247 5521 raft_consensus.cc:2229] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:24.033618 5521 raft_consensus.cc:2258] T c1df93c7ecfa4c92ac07b5a31fcad69c P 4d591fe156374503b96092336507b7b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:24.525286 6617 consensus_queue.cc:579] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [LEADER]: The tablet replica hosted on peer 4d591fe156374503b96092336507b7b4 has failed
W20260810 00:03:24.525841 6617 consensus_peers.cc:487] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d -> Peer 4d591fe156374503b96092336507b7b4 (127.5.100.65:42357): Couldn't send request to peer 4d591fe156374503b96092336507b7b4. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:24.556399 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:24.575018 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:24.591960 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:24.599265 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:03:24.615490 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:24.616024 5521 tablet_replica.cc:330] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d: stopping tablet replica
I20260810 00:03:24.616925 5521 raft_consensus.cc:2229] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:24.617655 5521 raft_consensus.cc:2258] T c1df93c7ecfa4c92ac07b5a31fcad69c P 38d5cf32084a48108771b955232a317d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:24.635382 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:24.642961 5521 tablet_server.cc:178] TabletServer@127.5.100.67:0 shutting down...
I20260810 00:03:24.659789 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:24.660513 5521 tablet_replica.cc:330] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480: stopping tablet replica
I20260810 00:03:24.661029 5521 raft_consensus.cc:2229] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:24.661422 5521 raft_consensus.cc:2258] T c1df93c7ecfa4c92ac07b5a31fcad69c P 9cc268cc6df8461c8585fb1f8817c480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:24.678773 5521 tablet_server.cc:194] TabletServer@127.5.100.67:0 shutdown complete.
I20260810 00:03:24.685904 5521 master.cc:537] Master@127.5.100.126:46527 shutting down...
I20260810 00:03:24.699333 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:24.699790 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:24.700088 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 607c1e6f945c4729874cd83b97790323: stopping tablet replica
I20260810 00:03:24.716521 5521 master.cc:552] Master@127.5.100.126:46527 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (2475 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260810 00:03:24.739308 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:41929
I20260810 00:03:24.740367 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:24.744810 6802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:24.746060 6803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:24.747233 6805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:24.747483 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:24.748101 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:24.748310 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:24.748409 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320204748402 us; error 0 us; skew 500 ppm
I20260810 00:03:24.748840 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:24.755497 5521 webserver.cc:460] Webserver started at http://127.5.100.126:39785/ using document root <none> and password file <none>
I20260810 00:03:24.755929 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:24.756093 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:24.756402 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:24.757464 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "51af2ca546db41d59389b2c0323ff47a"
format_stamp: "Formatted at 2026-08-10 00:03:24 on dist-test-slave-8w9v"
I20260810 00:03:24.761914 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 00:03:24.765081 6810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:24.765777 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:03:24.766016 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "51af2ca546db41d59389b2c0323ff47a"
format_stamp: "Formatted at 2026-08-10 00:03:24 on dist-test-slave-8w9v"
I20260810 00:03:24.766268 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:24.784896 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:24.785748 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:24.821373 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:41929
I20260810 00:03:24.821461 6864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:41929 every 8 connection(s)
I20260810 00:03:24.825011 6865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:24.834741 6865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a: Bootstrap starting.
I20260810 00:03:24.838932 6865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:24.842547 6865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a: No bootstrap required, opened a new log
I20260810 00:03:24.844317 6865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER }
I20260810 00:03:24.844736 6865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:24.844969 6865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51af2ca546db41d59389b2c0323ff47a, State: Initialized, Role: FOLLOWER
I20260810 00:03:24.845575 6865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER }
I20260810 00:03:24.846032 6865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:24.846264 6865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:24.846544 6865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:24.927842 6865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER }
I20260810 00:03:24.928612 6865 leader_election.cc:302] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51af2ca546db41d59389b2c0323ff47a; no voters:
I20260810 00:03:24.929932 6865 leader_election.cc:288] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:24.930351 6868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:24.931973 6868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 1 LEADER]: Becoming Leader. State: Replica: 51af2ca546db41d59389b2c0323ff47a, State: Running, Role: LEADER
I20260810 00:03:24.932866 6868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER }
I20260810 00:03:24.933490 6865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:24.935588 6869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51af2ca546db41d59389b2c0323ff47a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER } }
I20260810 00:03:24.935588 6871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51af2ca546db41d59389b2c0323ff47a. Latest consensus state: current_term: 1 leader_uuid: "51af2ca546db41d59389b2c0323ff47a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51af2ca546db41d59389b2c0323ff47a" member_type: VOTER } }
I20260810 00:03:24.936216 6869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:24.936246 6871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:24.939816 6874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:24.945799 6874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:24.947089 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:24.953740 6874 catalog_manager.cc:1347] Generated new cluster ID: fa155e63057a4c5d87751be43791e46f
I20260810 00:03:24.953927 6874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:24.967406 6874 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:24.969094 6874 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:24.980396 6874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a: Generated new TSK 0
I20260810 00:03:24.980927 6874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:25.013237 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:25.018786 6890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.020954 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.022869 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:25.022763 6895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.023842 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.024055 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.024330 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205024312 us; error 0 us; skew 500 ppm
I20260810 00:03:25.024924 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.027364 5521 webserver.cc:460] Webserver started at http://127.5.100.65:39797/ using document root <none> and password file <none>
I20260810 00:03:25.027841 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.028040 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.028395 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.029533 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "13c5db7dd35a429d8d901c1598287145"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.034235 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:03:25.037700 6903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.038362 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:03:25.038699 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "13c5db7dd35a429d8d901c1598287145"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.039052 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.049973 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.051541 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.052945 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:25.055015 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:25.055227 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.055464 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:25.055631 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.094141 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:37979
I20260810 00:03:25.094213 6968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:37979 every 8 connection(s)
I20260810 00:03:25.109221 6969 heartbeater.cc:347] Connected to a master server at 127.5.100.126:41929
I20260810 00:03:25.109612 6969 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:25.110212 6969 heartbeater.cc:511] Master 127.5.100.126:41929 requested a full tablet report, sending...
I20260810 00:03:25.112069 6828 ts_manager.cc:194] Registered new tserver with Master: 13c5db7dd35a429d8d901c1598287145 (127.5.100.65:37979)
I20260810 00:03:25.112861 5521 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.015413705s
I20260810 00:03:25.114008 6828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49898
I20260810 00:03:25.135699 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:25.153402 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:25.169275 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:25.176522 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:03:25.179105 5521 tablet_server.cc:93] deduplicated master addresses: 127.5.100.126:41929
W20260810 00:03:25.182188 6981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.183751 6982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.183948 6984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.185245 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:25.186120 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.186321 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.186487 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205186456 us; error 0 us; skew 500 ppm
I20260810 00:03:25.187026 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.189693 5521 webserver.cc:460] Webserver started at http://127.5.100.65:39797/ using document root <none> and password file <none>
I20260810 00:03:25.190181 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.190351 5521 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260810 00:03:25.194005 5521 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 00:03:25.196473 6989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.197147 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:03:25.197440 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "13c5db7dd35a429d8d901c1598287145"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.197708 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.214715 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.215838 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.217314 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:25.220335 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:25.220498 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.220770 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:25.220919 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.262073 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:37979
I20260810 00:03:25.262184 7059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:37979 every 8 connection(s)
I20260810 00:03:25.267870 5521 tablet_server.cc:178] TabletServer@127.5.100.65:37979 shutting down...
I20260810 00:03:25.289176 7060 heartbeater.cc:347] Connected to a master server at 127.5.100.126:41929
I20260810 00:03:25.289644 7060 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:25.290421 7060 heartbeater.cc:511] Master 127.5.100.126:41929 requested a full tablet report, sending...
I20260810 00:03:25.293668 6828 ts_manager.cc:194] Re-registered known tserver with Master: 13c5db7dd35a429d8d901c1598287145 (127.5.100.65:37979)
I20260810 00:03:25.295593 6828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49918
I20260810 00:03:25.298261 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:25.314038 5521 tablet_server.cc:194] TabletServer@127.5.100.65:37979 shutdown complete.
I20260810 00:03:25.321095 5521 master.cc:537] Master@127.5.100.126:41929 shutting down...
I20260810 00:03:25.334471 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:25.335551 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:25.335978 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 51af2ca546db41d59389b2c0323ff47a: stopping tablet replica
I20260810 00:03:25.356902 5521 master.cc:552] Master@127.5.100.126:41929 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (637 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260810 00:03:25.377784 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:33747,127.5.100.125:42269,127.5.100.124:40327
I20260810 00:03:25.378860 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:25.383430 7068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.384850 7069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.386418 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:25.386345 7071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.387280 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.387449 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.387552 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205387544 us; error 0 us; skew 500 ppm
I20260810 00:03:25.387965 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.390177 5521 webserver.cc:460] Webserver started at http://127.5.100.126:34163/ using document root <none> and password file <none>
I20260810 00:03:25.390592 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.390764 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.390967 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.391899 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "1a165b9cb2d1422ca7693fc2287e0471"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.396180 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 00:03:25.399186 7076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.399971 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:03:25.400256 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "1a165b9cb2d1422ca7693fc2287e0471"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.400549 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.414469 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.415472 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.454260 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:33747
I20260810 00:03:25.454383 7129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:33747 every 8 connection(s)
I20260810 00:03:25.458042 7130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:25.458312 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:25.463855 7132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.464368 7133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.466008 7130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } has no permanent_uuid. Determining permanent_uuid...
W20260810 00:03:25.467165 7136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.468256 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:25.469487 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.469732 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.469907 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205469887 us; error 0 us; skew 500 ppm
I20260810 00:03:25.470674 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.473541 5521 webserver.cc:460] Webserver started at http://127.5.100.125:41955/ using document root <none> and password file <none>
I20260810 00:03:25.474144 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.474431 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.474766 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.476083 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-1-root/instance:
uuid: "60e7224900864daba37bc2346bf79e94"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.481424 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 00:03:25.483623 7130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.485208 7143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.486091 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
W20260810 00:03:25.486071 7077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.100.125:42269: connect: Connection refused (error 111)
I20260810 00:03:25.486501 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-1-root
uuid: "60e7224900864daba37bc2346bf79e94"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.486804 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260810 00:03:25.489117 7130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.100.125:42269: Network error: Client connection negotiation failed: client connection to 127.5.100.125:42269: connect: Connection refused (error 111)
I20260810 00:03:25.505920 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.506906 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.540686 7130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } attempt: 1
I20260810 00:03:25.544148 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.125:42269
I20260810 00:03:25.544246 7200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.125:42269 every 8 connection(s)
I20260810 00:03:25.548956 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:03:25.549041 7202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 00:03:25.555455 7205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.557155 7206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.558605 7202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.559681 7130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.566958 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:25.567402 7130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.100.124:40327: Network error: Client connection negotiation failed: client connection to 127.5.100.124:40327: connect: Connection refused (error 111)
W20260810 00:03:25.568822 7208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.569659 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.569855 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.570080 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205570036 us; error 0 us; skew 500 ppm
I20260810 00:03:25.570636 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.573592 5521 webserver.cc:460] Webserver started at http://127.5.100.124:44601/ using document root <none> and password file <none>
I20260810 00:03:25.574230 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.574491 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.574824 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.576503 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-2-root/instance:
uuid: "065a70ec01b04c339834bd751671ad23"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.576633 7202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.582538 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 00:03:25.586907 7215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.587884 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:03:25.588433 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-2-root
uuid: "065a70ec01b04c339834bd751671ad23"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.588786 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.589632 7202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } has no permanent_uuid. Determining permanent_uuid...
W20260810 00:03:25.595124 7202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.100.124:40327: Network error: Client connection negotiation failed: client connection to 127.5.100.124:40327: connect: Connection refused (error 111)
I20260810 00:03:25.610232 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.611171 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.613018 7130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } attempt: 1
I20260810 00:03:25.613656 7202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } attempt: 1
W20260810 00:03:25.617354 7130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.100.124:40327: Network error: Client connection negotiation failed: client connection to 127.5.100.124:40327: connect: Connection refused (error 111)
W20260810 00:03:25.617846 7202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.100.124:40327: Network error: Client connection negotiation failed: client connection to 127.5.100.124:40327: connect: Connection refused (error 111)
I20260810 00:03:25.647362 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.124:40327
I20260810 00:03:25.647487 7274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.124:40327 every 8 connection(s)
I20260810 00:03:25.650205 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:25.651255 7275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:25.656754 7275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.658322 7202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } attempt: 2
I20260810 00:03:25.671837 7130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } attempt: 2
I20260810 00:03:25.673674 7275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.681075 7202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: Bootstrap starting.
I20260810 00:03:25.686978 7202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:25.690248 7130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: Bootstrap starting.
I20260810 00:03:25.692673 7202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: No bootstrap required, opened a new log
I20260810 00:03:25.694020 7275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } has no permanent_uuid. Determining permanent_uuid...
I20260810 00:03:25.695201 7130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:25.695113 7202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.695935 7202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:25.696269 7202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e7224900864daba37bc2346bf79e94, State: Initialized, Role: FOLLOWER
I20260810 00:03:25.697058 7202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.699218 7282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.700105 7282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.700937 7202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:25.706280 7130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: No bootstrap required, opened a new log
I20260810 00:03:25.708302 7130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.708923 7130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:25.709197 7130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a165b9cb2d1422ca7693fc2287e0471, State: Initialized, Role: FOLLOWER
I20260810 00:03:25.709914 7130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.720510 7275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: Bootstrap starting.
I20260810 00:03:25.720901 7286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.721757 7286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.722538 7130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 00:03:25.723933 7295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:03:25.724320 7295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:03:25.726675 7275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:25.731856 7275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: No bootstrap required, opened a new log
I20260810 00:03:25.734512 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260810 00:03:25.734845 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260810 00:03:25.734624 7275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.735394 7275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:25.735634 7275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 065a70ec01b04c339834bd751671ad23, State: Initialized, Role: FOLLOWER
I20260810 00:03:25.736207 7275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
W20260810 00:03:25.737847 7308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:03:25.738139 7308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 00:03:25.738003 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.738945 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.739549 7275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:25.752462 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:25.754899 7321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 00:03:25.755160 7321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 00:03:25.758848 7322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.759305 7323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.760601 7309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:25.761123 7309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
W20260810 00:03:25.761916 7325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.763159 7309 leader_election.cc:288] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269)
I20260810 00:03:25.763731 7282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:25.763871 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471" is_pre_election: true
I20260810 00:03:25.764196 7282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.764617 7105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 0.
I20260810 00:03:25.764670 7175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94" is_pre_election: true
I20260810 00:03:25.765539 7175 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 0.
I20260810 00:03:25.765787 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:25.765910 7219 leader_election.cc:302] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471; no voters:
I20260810 00:03:25.766270 7282 leader_election.cc:288] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:25.767035 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.767279 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.767455 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205767439 us; error 0 us; skew 500 ppm
I20260810 00:03:25.767576 7309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:25.767710 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23" is_pre_election: true
I20260810 00:03:25.767935 7309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:25.768055 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.768281 7309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:25.768600 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471" is_pre_election: true
I20260810 00:03:25.769150 7105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60e7224900864daba37bc2346bf79e94 in term 0.
I20260810 00:03:25.770216 7148 leader_election.cc:302] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a165b9cb2d1422ca7693fc2287e0471, 60e7224900864daba37bc2346bf79e94; no voters:
I20260810 00:03:25.771085 7282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:25.771260 5521 webserver.cc:460] Webserver started at http://127.5.100.65:45337/ using document root <none> and password file <none>
I20260810 00:03:25.771459 7282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:25.771821 7282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:25.771943 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.772264 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.772557 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.773597 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "52b0468e66084ff59c01fb5b78d3f627"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.774948 7309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.776162 7247 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60e7224900864daba37bc2346bf79e94 in current term 1: Already voted for candidate 065a70ec01b04c339834bd751671ad23 in this term.
I20260810 00:03:25.777786 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471"
I20260810 00:03:25.778465 7105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:25.779157 7175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94"
I20260810 00:03:25.779776 7309 leader_election.cc:288] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 1 election: Requested vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269)
I20260810 00:03:25.783442 7105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 1.
I20260810 00:03:25.784315 7219 leader_election.cc:302] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471; no voters:
I20260810 00:03:25.784736 7282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.785171 7309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:25.785435 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.003s
I20260810 00:03:25.785959 7175 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 065a70ec01b04c339834bd751671ad23 in current term 1: Already voted for candidate 60e7224900864daba37bc2346bf79e94 in this term.
I20260810 00:03:25.787853 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471"
I20260810 00:03:25.788300 7309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 LEADER]: Becoming Leader. State: Replica: 065a70ec01b04c339834bd751671ad23, State: Running, Role: LEADER
I20260810 00:03:25.788692 7282 leader_election.cc:288] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 1 election: Requested vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:25.789448 7105 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60e7224900864daba37bc2346bf79e94 in current term 1: Already voted for candidate 065a70ec01b04c339834bd751671ad23 in this term.
I20260810 00:03:25.789703 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23"
I20260810 00:03:25.789592 7309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:25.790916 7147 leader_election.cc:302] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60e7224900864daba37bc2346bf79e94; no voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471
I20260810 00:03:25.791808 7282 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 00:03:25.792052 7333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.793041 5521 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 00:03:25.793385 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "52b0468e66084ff59c01fb5b78d3f627"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.793709 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.793939 7332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.794716 7332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:25.796077 7335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:25.801294 7335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:25.804903 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.805908 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.807298 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:25.809139 7175 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Refusing update from remote peer 065a70ec01b04c339834bd751671ad23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:03:25.809662 7105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Refusing update from remote peer 065a70ec01b04c339834bd751671ad23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:03:25.810155 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:25.810448 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.810756 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:25.810521 7332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:25.810967 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.811275 7309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:25.825903 7286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.826829 7286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.830753 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.831693 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:25.832005 7335 catalog_manager.cc:1347] Generated new cluster ID: a45a95041b3d41e395141d73b369ad7f
I20260810 00:03:25.832381 7335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:25.836530 7286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.837141 7286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.838994 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.839627 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:25.841213 7282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.841784 7282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.844786 7282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:25.845474 7282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:25.874882 7335 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:25.877096 7335 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:25.897699 7335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: Generated new TSK 0
I20260810 00:03:25.898445 7335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:25.908496 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:46289
I20260810 00:03:25.908592 7415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:46289 every 8 connection(s)
I20260810 00:03:25.916927 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:25.933652 7423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:25.943059 7424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.943292 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:25.943828 7417 heartbeater.cc:347] Connected to a master server at 127.5.100.125:42269
I20260810 00:03:25.944304 7417 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:25.945189 7417 heartbeater.cc:511] Master 127.5.100.125:42269 requested a full tablet report, sending...
W20260810 00:03:25.947162 7426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:25.948318 7163 ts_manager.cc:194] Registered new tserver with Master: 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289)
I20260810 00:03:25.948333 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:25.949787 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:25.950019 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320205950003 us; error 0 us; skew 500 ppm
I20260810 00:03:25.950793 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:25.950845 7416 heartbeater.cc:347] Connected to a master server at 127.5.100.124:40327
I20260810 00:03:25.951282 7416 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:25.952131 7416 heartbeater.cc:511] Master 127.5.100.124:40327 requested a full tablet report, sending...
I20260810 00:03:25.953405 7418 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33747
I20260810 00:03:25.953799 7418 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:25.954010 5521 webserver.cc:460] Webserver started at http://127.5.100.66:37525/ using document root <none> and password file <none>
I20260810 00:03:25.954576 7418 heartbeater.cc:511] Master 127.5.100.126:33747 requested a full tablet report, sending...
I20260810 00:03:25.954691 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:25.954653 7236 ts_manager.cc:194] Registered new tserver with Master: 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289)
I20260810 00:03:25.955060 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:25.955399 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:25.956837 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "bb0187aed98945ab8e104a9fdf3d4401"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.957058 7094 ts_manager.cc:194] Registered new tserver with Master: 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289)
I20260810 00:03:25.957407 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37022
I20260810 00:03:25.961513 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:03:25.964759 7433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.965495 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:03:25.965754 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "bb0187aed98945ab8e104a9fdf3d4401"
format_stamp: "Formatted at 2026-08-10 00:03:25 on dist-test-slave-8w9v"
I20260810 00:03:25.965984 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:25.978586 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:25.979645 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:25.981680 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:25.985183 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:25.985411 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:25.985641 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:25.985808 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:26.060179 7416 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:26.068378 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:39541
I20260810 00:03:26.068498 7501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:39541 every 8 connection(s)
I20260810 00:03:26.083077 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:26.099143 7510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:26.102521 7502 heartbeater.cc:347] Connected to a master server at 127.5.100.124:40327
I20260810 00:03:26.103037 7502 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.104321 7502 heartbeater.cc:511] Master 127.5.100.124:40327 requested a full tablet report, sending...
I20260810 00:03:26.106899 7236 ts_manager.cc:194] Registered new tserver with Master: bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:26.107517 7503 heartbeater.cc:347] Connected to a master server at 127.5.100.125:42269
I20260810 00:03:26.108018 7503 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.109212 7503 heartbeater.cc:511] Master 127.5.100.125:42269 requested a full tablet report, sending...
I20260810 00:03:26.110221 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37030
W20260810 00:03:26.111616 7511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:26.112401 7163 ts_manager.cc:194] Registered new tserver with Master: bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:26.117486 7505 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33747
I20260810 00:03:26.117998 7505 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.118845 7505 heartbeater.cc:511] Master 127.5.100.126:33747 requested a full tablet report, sending...
I20260810 00:03:26.121835 7094 ts_manager.cc:194] Registered new tserver with Master: bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:26.122685 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:26.122888 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:26.123988 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:26.124300 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:26.124495 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320206124479 us; error 0 us; skew 500 ppm
I20260810 00:03:26.125021 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:26.127147 5521 webserver.cc:460] Webserver started at http://127.5.100.67:39483/ using document root <none> and password file <none>
I20260810 00:03:26.127627 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:26.127839 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:26.128158 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:26.129304 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-2-root/instance:
uuid: "2b713c5b20574640948496d076483447"
format_stamp: "Formatted at 2026-08-10 00:03:26 on dist-test-slave-8w9v"
I20260810 00:03:26.133404 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:03:26.136518 7522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:26.137203 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:03:26.137514 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-2-root
uuid: "2b713c5b20574640948496d076483447"
format_stamp: "Formatted at 2026-08-10 00:03:26 on dist-test-slave-8w9v"
I20260810 00:03:26.137802 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:26.160638 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:26.162091 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:26.164758 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:26.169008 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:26.169203 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:26.169446 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:26.169595 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:26.214989 7502 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:26.248507 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.67:44659
I20260810 00:03:26.248620 7586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.67:44659 every 8 connection(s)
I20260810 00:03:26.277707 7587 heartbeater.cc:347] Connected to a master server at 127.5.100.124:40327
I20260810 00:03:26.278155 7587 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.278915 7587 heartbeater.cc:511] Master 127.5.100.124:40327 requested a full tablet report, sending...
I20260810 00:03:26.281184 7236 ts_manager.cc:194] Registered new tserver with Master: 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.283151 7589 heartbeater.cc:347] Connected to a master server at 127.5.100.125:42269
I20260810 00:03:26.283318 7591 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33747
I20260810 00:03:26.283737 7589 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.283856 7591 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:26.283201 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37042
I20260810 00:03:26.284876 7589 heartbeater.cc:511] Master 127.5.100.125:42269 requested a full tablet report, sending...
I20260810 00:03:26.284914 7591 heartbeater.cc:511] Master 127.5.100.126:33747 requested a full tablet report, sending...
I20260810 00:03:26.287726 7163 ts_manager.cc:194] Registered new tserver with Master: 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.288178 7094 ts_manager.cc:194] Registered new tserver with Master: 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.289017 5521 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.027727885s
I20260810 00:03:26.329660 7235 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260810 00:03:26.331284 7235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:03:26.387926 7587 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:26.401631 7462 tablet_service.cc:1467] Processing CreateTablet for tablet da63227906254ceb8195cd2b134a5fd6 (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.403213 7462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da63227906254ceb8195cd2b134a5fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.405014 7461 tablet_service.cc:1467] Processing CreateTablet for tablet 84aabfe16f1d4823bcc38f6ff128dffd (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.406230 7461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84aabfe16f1d4823bcc38f6ff128dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.413470 7460 tablet_service.cc:1467] Processing CreateTablet for tablet 039aeddf930c46c6bb2230b9e95d229d (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.414871 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039aeddf930c46c6bb2230b9e95d229d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.423652 7550 tablet_service.cc:1467] Processing CreateTablet for tablet 039aeddf930c46c6bb2230b9e95d229d (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.424685 7551 tablet_service.cc:1467] Processing CreateTablet for tablet 84aabfe16f1d4823bcc38f6ff128dffd (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.422792 7377 tablet_service.cc:1467] Processing CreateTablet for tablet 84aabfe16f1d4823bcc38f6ff128dffd (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.426616 7550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039aeddf930c46c6bb2230b9e95d229d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.423741 7376 tablet_service.cc:1467] Processing CreateTablet for tablet 039aeddf930c46c6bb2230b9e95d229d (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.420408 7378 tablet_service.cc:1467] Processing CreateTablet for tablet da63227906254ceb8195cd2b134a5fd6 (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.428973 7376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039aeddf930c46c6bb2230b9e95d229d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.424584 7552 tablet_service.cc:1467] Processing CreateTablet for tablet da63227906254ceb8195cd2b134a5fd6 (DEFAULT_TABLE table=test-table [id=4e0625bb239f40b4a63d14bd2d305504]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260810 00:03:26.430891 7378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da63227906254ceb8195cd2b134a5fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.434060 7552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da63227906254ceb8195cd2b134a5fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.436103 7377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84aabfe16f1d4823bcc38f6ff128dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.437026 7551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84aabfe16f1d4823bcc38f6ff128dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:26.450443 7611 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: Bootstrap starting.
I20260810 00:03:26.457131 7611 tablet_bootstrap.cc:654] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.475857 7611 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: No bootstrap required, opened a new log
I20260810 00:03:26.476433 7611 ts_tablet_manager.cc:1397] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: Time spent bootstrapping tablet: real 0.026s user 0.013s sys 0.009s
I20260810 00:03:26.477900 7613 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: Bootstrap starting.
I20260810 00:03:26.483484 7611 raft_consensus.cc:348] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.484601 7611 raft_consensus.cc:374] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.484927 7611 raft_consensus.cc:729] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.486183 7613 tablet_bootstrap.cc:654] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.485745 7611 consensus_queue.cc:260] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.490236 7611 ts_tablet_manager.cc:1428] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: Time spent starting tablet: real 0.013s user 0.006s sys 0.002s
I20260810 00:03:26.492759 7611 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: Bootstrap starting.
I20260810 00:03:26.494405 7614 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: Bootstrap starting.
I20260810 00:03:26.506193 7611 tablet_bootstrap.cc:654] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.525022 7614 tablet_bootstrap.cc:654] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.529534 7613 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: No bootstrap required, opened a new log
I20260810 00:03:26.530021 7611 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: No bootstrap required, opened a new log
I20260810 00:03:26.530099 7613 ts_tablet_manager.cc:1397] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.027s
I20260810 00:03:26.530773 7611 ts_tablet_manager.cc:1397] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: Time spent bootstrapping tablet: real 0.039s user 0.018s sys 0.007s
I20260810 00:03:26.533182 7614 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: No bootstrap required, opened a new log
I20260810 00:03:26.533542 7613 raft_consensus.cc:348] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.534086 7614 ts_tablet_manager.cc:1397] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: Time spent bootstrapping tablet: real 0.051s user 0.015s sys 0.000s
I20260810 00:03:26.533845 7611 raft_consensus.cc:348] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.534278 7613 raft_consensus.cc:374] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.534636 7611 raft_consensus.cc:374] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.534703 7613 raft_consensus.cc:729] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b0468e66084ff59c01fb5b78d3f627, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.535092 7611 raft_consensus.cc:729] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.535604 7613 consensus_queue.cc:260] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.535836 7611 consensus_queue.cc:260] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.536881 7614 raft_consensus.cc:348] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.537632 7614 raft_consensus.cc:374] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.537912 7614 raft_consensus.cc:729] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b713c5b20574640948496d076483447, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.538158 7611 ts_tablet_manager.cc:1428] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 00:03:26.538744 7614 consensus_queue.cc:260] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.540148 7611 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: Bootstrap starting.
I20260810 00:03:26.546738 7613 ts_tablet_manager.cc:1428] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: Time spent starting tablet: real 0.016s user 0.010s sys 0.002s
I20260810 00:03:26.548933 7611 tablet_bootstrap.cc:654] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.553989 7615 raft_consensus.cc:482] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.554752 7613 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: Bootstrap starting.
I20260810 00:03:26.554481 7615 raft_consensus.cc:504] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.564627 7620 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.565127 7620 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.570004 7614 ts_tablet_manager.cc:1428] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: Time spent starting tablet: real 0.036s user 0.019s sys 0.000s
I20260810 00:03:26.580348 7614 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: Bootstrap starting.
I20260810 00:03:26.585150 7615 leader_election.cc:288] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.596496 7620 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.598127 7614 tablet_bootstrap.cc:654] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.598145 7613 tablet_bootstrap.cc:654] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.618006 7621 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.618527 7621 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.622715 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:26.623344 7611 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: No bootstrap required, opened a new log
I20260810 00:03:26.625650 7611 ts_tablet_manager.cc:1397] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: Time spent bootstrapping tablet: real 0.086s user 0.012s sys 0.041s
I20260810 00:03:26.625519 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:26.626379 7621 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
W20260810 00:03:26.627224 7436 leader_election.cc:341] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289): Illegal state: must be running to vote when last-logged opid is not known
I20260810 00:03:26.628087 7615 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.631244 7615 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.633927 7615 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.634107 7613 tablet_bootstrap.cc:492] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: No bootstrap required, opened a new log
I20260810 00:03:26.634238 7614 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: No bootstrap required, opened a new log
W20260810 00:03:26.632190 7434 leader_election.cc:341] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b713c5b20574640948496d076483447 (127.5.100.67:44659): Illegal state: must be running to vote when last-logged opid is not known
I20260810 00:03:26.634513 7613 ts_tablet_manager.cc:1397] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.010s
I20260810 00:03:26.634989 7614 ts_tablet_manager.cc:1397] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.013s
I20260810 00:03:26.634917 7434 leader_election.cc:302] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb0187aed98945ab8e104a9fdf3d4401; no voters: 2b713c5b20574640948496d076483447, 52b0468e66084ff59c01fb5b78d3f627
I20260810 00:03:26.637387 7615 raft_consensus.cc:2726] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 00:03:26.639237 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:26.639639 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:26.639910 7389 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 0.
I20260810 00:03:26.639853 7613 raft_consensus.cc:348] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.640762 7613 raft_consensus.cc:374] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 00:03:26.640889 7434 leader_election.cc:341] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b713c5b20574640948496d076483447 (127.5.100.67:44659): Illegal state: must be running to vote when last-logged opid is not known
I20260810 00:03:26.641067 7613 raft_consensus.cc:729] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b0468e66084ff59c01fb5b78d3f627, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.641515 7436 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401; no voters: 2b713c5b20574640948496d076483447
I20260810 00:03:26.641930 7613 consensus_queue.cc:260] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.642753 7615 raft_consensus.cc:2781] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:26.643116 7615 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:26.643466 7615 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.637686 7611 raft_consensus.cc:348] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.645516 7611 raft_consensus.cc:374] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.645835 7611 raft_consensus.cc:729] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.646543 7611 consensus_queue.cc:260] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.647989 7613 ts_tablet_manager.cc:1428] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20260810 00:03:26.649041 7613 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: Bootstrap starting.
I20260810 00:03:26.649279 7614 raft_consensus.cc:348] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.652707 7614 raft_consensus.cc:374] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.653107 7614 raft_consensus.cc:729] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b713c5b20574640948496d076483447, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.658670 7611 ts_tablet_manager.cc:1428] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: Time spent starting tablet: real 0.032s user 0.010s sys 0.005s
I20260810 00:03:26.654210 7614 consensus_queue.cc:260] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.660383 7613 tablet_bootstrap.cc:654] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.664290 7621 raft_consensus.cc:482] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.664764 7621 raft_consensus.cc:504] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.667359 7621 leader_election.cc:288] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:26.670327 7614 ts_tablet_manager.cc:1428] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: Time spent starting tablet: real 0.034s user 0.007s sys 0.001s
I20260810 00:03:26.678222 7614 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: Bootstrap starting.
I20260810 00:03:26.693147 7631 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.694324 7631 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.701613 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:26.702282 7562 raft_consensus.cc:2454] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 0.
I20260810 00:03:26.705670 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:26.706111 7434 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401; no voters:
I20260810 00:03:26.705691 7620 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:26.699985 7631 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.704114 7614 tablet_bootstrap.cc:654] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:26.701604 7615 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.706862 7620 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.712589 7389 raft_consensus.cc:2454] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 0.
I20260810 00:03:26.714075 7615 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 election: Requested vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.713943 7620 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.716075 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627"
I20260810 00:03:26.717479 7388 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.710457 7631 raft_consensus.cc:2781] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:26.718568 7631 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:26.721793 7631 raft_consensus.cc:3037] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.723496 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447"
W20260810 00:03:26.732734 7434 leader_election.cc:341] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2b713c5b20574640948496d076483447 (127.5.100.67:44659): Illegal state: must be running to vote when last-logged opid is not known
I20260810 00:03:26.756816 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.758249 7470 raft_consensus.cc:2454] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b713c5b20574640948496d076483447 in term 0.
I20260810 00:03:26.750203 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.748404 7474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.761124 7474 raft_consensus.cc:2379] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b0468e66084ff59c01fb5b78d3f627 in current term 1: Already voted for candidate bb0187aed98945ab8e104a9fdf3d4401 in this term.
I20260810 00:03:26.764379 7523 leader_election.cc:302] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401; no voters:
I20260810 00:03:26.765551 7621 raft_consensus.cc:2781] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:26.765873 7621 raft_consensus.cc:482] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:26.766186 7621 raft_consensus.cc:3037] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.792750 7388 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 1.
I20260810 00:03:26.814220 7295 catalog_manager.cc:1259] Loaded cluster ID: a45a95041b3d41e395141d73b369ad7f
I20260810 00:03:26.814657 7295 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: loading cluster ID for follower catalog manager: success
I20260810 00:03:26.929994 7436 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401; no voters: 2b713c5b20574640948496d076483447
I20260810 00:03:26.933176 7471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.935762 7295 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: acquiring CA information for follower catalog manager: success
I20260810 00:03:26.936015 7613 tablet_bootstrap.cc:492] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: No bootstrap required, opened a new log
I20260810 00:03:26.919369 7621 raft_consensus.cc:504] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.945174 7614 tablet_bootstrap.cc:492] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: No bootstrap required, opened a new log
I20260810 00:03:26.945575 7614 ts_tablet_manager.cc:1397] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: Time spent bootstrapping tablet: real 0.268s user 0.002s sys 0.021s
I20260810 00:03:26.940349 7613 ts_tablet_manager.cc:1397] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: Time spent bootstrapping tablet: real 0.291s user 0.020s sys 0.030s
I20260810 00:03:26.947299 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 065a70ec01b04c339834bd751671ad23)
I20260810 00:03:26.950191 7633 raft_consensus.cc:2781] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:26.950296 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:26.950784 7633 raft_consensus.cc:686] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 LEADER]: Becoming Leader. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Running, Role: LEADER
I20260810 00:03:26.951611 7633 consensus_queue.cc:237] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.952723 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:26.948722 7295 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:03:26.948376 7614 raft_consensus.cc:348] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
W20260810 00:03:26.953238 7336 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.100.66:39541, user_credentials={real_user=slave}} blocked reactor thread for 101504us
I20260810 00:03:26.953608 7614 raft_consensus.cc:374] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.953977 7621 leader_election.cc:288] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 election: Requested vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:26.954150 7614 raft_consensus.cc:729] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b713c5b20574640948496d076483447, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.955017 7614 consensus_queue.cc:260] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.955708 7631 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.953104 7175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94" is_pre_election: true
I20260810 00:03:26.956960 7473 raft_consensus.cc:2379] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b713c5b20574640948496d076483447 in current term 1: Already voted for candidate bb0187aed98945ab8e104a9fdf3d4401 in this term.
I20260810 00:03:26.957086 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:26.952479 7613 raft_consensus.cc:348] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.957448 7614 ts_tablet_manager.cc:1428] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: Time spent starting tablet: real 0.012s user 0.001s sys 0.006s
I20260810 00:03:26.958747 7562 raft_consensus.cc:2454] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b0468e66084ff59c01fb5b78d3f627 in term 0.
I20260810 00:03:26.958832 7561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:26.962581 7637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 065a70ec01b04c339834bd751671ad23)
I20260810 00:03:26.963063 7561 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b0468e66084ff59c01fb5b78d3f627 in term 0.
W20260810 00:03:26.959079 7473 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60894 (request call id 0) took 210 ms (client timeout 264 ms). Trace:
I20260810 00:03:26.963260 7637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:26.958320 7613 raft_consensus.cc:374] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:26.967104 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627"
I20260810 00:03:26.968261 7471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401"
I20260810 00:03:26.968770 7471 raft_consensus.cc:3037] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 0 FOLLOWER]: Advancing to term 1
W20260810 00:03:26.964195 7473 rpcz_store.cc:269] 0810 00:03:26.748781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0810 00:03:26.748949 (+ 168us) service_pool.cc:227] Handling call
0810 00:03:26.958847 (+209898us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54003a02d4. stack: 00007f56fc10d033 00007f56fc10cf30 00007f56fba6b534 00000000003b4464 00000000003b43f9 00007f5702931471 00007f57029b02ae 00007f5704bb9c27 00007f5702541c04 00007f5702541b92 00007f5702541ac1 00007f5702541a3c 00007f5702540cf2 00007f5700c6d94c 00007f5700c6cd86 00007f5700c6c72f
0810 00:03:26.959010 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":452530048}
I20260810 00:03:26.966872 7613 raft_consensus.cc:729] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b0468e66084ff59c01fb5b78d3f627, State: Initialized, Role: FOLLOWER
I20260810 00:03:26.970293 7613 consensus_queue.cc:260] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.971146 7388 raft_consensus.cc:3037] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.973489 7308 catalog_manager.cc:1259] Loaded cluster ID: a45a95041b3d41e395141d73b369ad7f
I20260810 00:03:26.973804 7308 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: loading cluster ID for follower catalog manager: success
I20260810 00:03:26.973878 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23" is_pre_election: true
I20260810 00:03:26.972748 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.976850 7175 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a165b9cb2d1422ca7693fc2287e0471 in term 1.
I20260810 00:03:26.975888 7336 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b713c5b20574640948496d076483447, 52b0468e66084ff59c01fb5b78d3f627; no voters: bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:26.979353 7620 raft_consensus.cc:2781] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:26.979687 7620 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:26.979830 7471 raft_consensus.cc:2454] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b713c5b20574640948496d076483447 in term 1.
I20260810 00:03:26.980031 7620 raft_consensus.cc:3037] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:26.980996 7637 leader_election.cc:288] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:26.981638 7077 leader_election.cc:302] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a165b9cb2d1422ca7693fc2287e0471, 60e7224900864daba37bc2346bf79e94; no voters:
I20260810 00:03:26.982513 7638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:03:26.982861 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 065a70ec01b04c339834bd751671ad23)
I20260810 00:03:26.983215 7638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:26.984531 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:26.985280 7473 raft_consensus.cc:2379] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b713c5b20574640948496d076483447 in current term 1: Already voted for candidate bb0187aed98945ab8e104a9fdf3d4401 in this term.
I20260810 00:03:26.987370 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471" is_pre_election: true
I20260810 00:03:26.979231 7336 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, 52b0468e66084ff59c01fb5b78d3f627; no voters:
I20260810 00:03:26.989027 7523 leader_election.cc:302] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401; no voters:
I20260810 00:03:26.989965 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:26.994278 7620 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:26.989995 7387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:26.996044 7387 raft_consensus.cc:2379] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b713c5b20574640948496d076483447 in current term 1: Already voted for candidate 52b0468e66084ff59c01fb5b78d3f627 in this term.
I20260810 00:03:26.996248 7620 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 election: Requested vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:26.987888 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23" is_pre_election: true
I20260810 00:03:26.998565 7147 leader_election.cc:302] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60e7224900864daba37bc2346bf79e94; no voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471
I20260810 00:03:26.991492 7308 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: acquiring CA information for follower catalog manager: success
I20260810 00:03:26.991536 7388 raft_consensus.cc:2454] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b713c5b20574640948496d076483447 in term 1.
I20260810 00:03:26.998836 7523 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b713c5b20574640948496d076483447; no voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:27.000489 7621 raft_consensus.cc:2781] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:27.001747 7637 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:03:27.006618 7308 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 00:03:27.006352 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:27.007728 7105 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 2 FOLLOWER]: Rejecting Update request from peer 065a70ec01b04c339834bd751671ad23 for earlier term 1. Current term is 2. Ops: []
I20260810 00:03:27.008543 7631 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 election: Requested vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:27.012897 7332 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 }, Status: INVALID_TERM, Last received: 1.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260810 00:03:27.015509 7561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447"
I20260810 00:03:27.016230 7561 raft_consensus.cc:3037] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:27.013101 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627"
I20260810 00:03:27.012195 7620 raft_consensus.cc:2741] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 00:03:27.020521 7388 raft_consensus.cc:2379] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb0187aed98945ab8e104a9fdf3d4401 in current term 1: Already voted for candidate 52b0468e66084ff59c01fb5b78d3f627 in this term.
I20260810 00:03:27.013156 7174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94"
I20260810 00:03:27.026652 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23"
I20260810 00:03:27.027015 7332 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 00:03:27.027347 7332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 065a70ec01b04c339834bd751671ad23, State: Running, Role: LEADER
I20260810 00:03:27.028195 7332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:27.031309 7078 leader_election.cc:302] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a165b9cb2d1422ca7693fc2287e0471; no voters: 065a70ec01b04c339834bd751671ad23, 60e7224900864daba37bc2346bf79e94
I20260810 00:03:27.024887 7621 raft_consensus.cc:686] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 1 LEADER]: Becoming Leader. State: Replica: 2b713c5b20574640948496d076483447, State: Running, Role: LEADER
I20260810 00:03:27.029294 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 2 election: Requested vote from peers 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:27.031837 7471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401"
I20260810 00:03:27.038195 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447"
I20260810 00:03:27.038645 7471 raft_consensus.cc:2379] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b0468e66084ff59c01fb5b78d3f627 in current term 1: Already voted for candidate bb0187aed98945ab8e104a9fdf3d4401 in this term.
I20260810 00:03:27.037853 7621 consensus_queue.cc:237] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:27.033977 7332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:27.043545 7235 catalog_manager.cc:5495] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66). New cstate: current_term: 1 leader_uuid: "bb0187aed98945ab8e104a9fdf3d4401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:27.049850 7336 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627; no voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:27.052079 7561 raft_consensus.cc:2454] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 1.
W20260810 00:03:27.053105 7598 master_proxy_rpc.cc:203] Re-attempting IsCreateTableDone request to leader Master (127.5.100.124:40327)
I20260810 00:03:27.053443 7434 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401; no voters: 52b0468e66084ff59c01fb5b78d3f627
I20260810 00:03:27.055568 7631 raft_consensus.cc:2781] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:27.057135 7631 raft_consensus.cc:686] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 LEADER]: Becoming Leader. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Running, Role: LEADER
I20260810 00:03:27.057646 7620 raft_consensus.cc:2726] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 00:03:27.047520 7638 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 00:03:27.059401 7613 ts_tablet_manager.cc:1428] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: Time spent starting tablet: real 0.109s user 0.008s sys 0.001s
I20260810 00:03:27.058944 7643 raft_consensus.cc:2726] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 00:03:27.060772 7631 consensus_queue.cc:237] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
E20260810 00:03:27.058568 7235 catalog_manager.cc:5650] Error updating tablets from bb0187aed98945ab8e104a9fdf3d4401: Illegal state: replica 065a70ec01b04c339834bd751671ad23 is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: true updated_tablets { tablet_id: "039aeddf930c46c6bb2230b9e95d229d" state: RUNNING consensus_state { current_term: 1 leader_uuid: "bb0187aed98945ab8e104a9fdf3d4401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } sequence_number: 11
I20260810 00:03:27.099399 7561 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:27.105118 7631 consensus_queue.cc:1035] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:27.106523 7631 consensus_queue.cc:1035] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:27.150342 7615 consensus_queue.cc:1035] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:27.152053 7643 consensus_queue.cc:1035] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:03:27.161286 7643 consensus_queue.cc:1035] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260810 00:03:27.161645 7615 consensus_queue.cc:1035] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260810 00:03:27.232637 7332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:27.233112 7332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:27.234833 7332 leader_election.cc:288] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269)
I20260810 00:03:27.235649 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471" is_pre_election: true
I20260810 00:03:27.235651 7174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94" is_pre_election: true
I20260810 00:03:27.236523 7105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 2.
I20260810 00:03:27.236660 7174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 1.
I20260810 00:03:27.237833 7216 leader_election.cc:302] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 065a70ec01b04c339834bd751671ad23, 60e7224900864daba37bc2346bf79e94; no voters:
I20260810 00:03:27.238684 7332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 00:03:27.239114 7332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:27.239444 7332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:27.244208 7332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:27.245641 7332 leader_election.cc:288] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 3 election: Requested vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269)
I20260810 00:03:27.246344 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471"
I20260810 00:03:27.246585 7174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "065a70ec01b04c339834bd751671ad23" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94"
I20260810 00:03:27.246928 7105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:27.247123 7174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 1 FOLLOWER]: Advancing to term 3
I20260810 00:03:27.253142 7105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 3.
I20260810 00:03:27.253142 7174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 065a70ec01b04c339834bd751671ad23 in term 3.
I20260810 00:03:27.254295 7219 leader_election.cc:302] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471; no voters:
I20260810 00:03:27.254927 7332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 00:03:27.255313 7332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 3 LEADER]: Becoming Leader. State: Replica: 065a70ec01b04c339834bd751671ad23, State: Running, Role: LEADER
I20260810 00:03:27.255877 7332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:27.261083 7642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.261658 7642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:27.262758 7662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:27.267807 7662 catalog_manager.cc:669] Loaded metadata for table test-table [id=4e0625bb239f40b4a63d14bd2d305504]
I20260810 00:03:27.273020 7662 tablet_loader.cc:96] loaded metadata for tablet 039aeddf930c46c6bb2230b9e95d229d (table test-table [id=4e0625bb239f40b4a63d14bd2d305504])
I20260810 00:03:27.274302 7662 tablet_loader.cc:96] loaded metadata for tablet 84aabfe16f1d4823bcc38f6ff128dffd (table test-table [id=4e0625bb239f40b4a63d14bd2d305504])
I20260810 00:03:27.275504 7662 tablet_loader.cc:96] loaded metadata for tablet da63227906254ceb8195cd2b134a5fd6 (table test-table [id=4e0625bb239f40b4a63d14bd2d305504])
I20260810 00:03:27.276888 7662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:27.280848 7662 catalog_manager.cc:1259] Loaded cluster ID: a45a95041b3d41e395141d73b369ad7f
I20260810 00:03:27.281105 7662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:27.284308 7662 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:27.286881 7662 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: Loaded TSK: 0
I20260810 00:03:27.288014 7662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:27.373338 7174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Refusing update from remote peer 065a70ec01b04c339834bd751671ad23: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260810 00:03:27.374766 7642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 00:03:27.380512 7637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.381203 7637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:27.382097 7105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Refusing update from remote peer 065a70ec01b04c339834bd751671ad23: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260810 00:03:27.383469 7332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 00:03:27.386595 7664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.387557 7664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:27.395134 7637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.395939 7637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:27.396755 7669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.396979 7638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065a70ec01b04c339834bd751671ad23. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.397460 7669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:27.397686 7638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 00:03:27.430087 7587 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:27.436398 7416 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:27.436487 7233 catalog_manager.cc:5495] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66). New cstate: current_term: 1 leader_uuid: "bb0187aed98945ab8e104a9fdf3d4401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } } }
I20260810 00:03:27.438287 7233 catalog_manager.cc:5495] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b713c5b20574640948496d076483447 (127.5.100.67). New cstate: current_term: 1 leader_uuid: "2b713c5b20574640948496d076483447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } health_report { overall_health: HEALTHY } } }
I20260810 00:03:27.439954 7233 catalog_manager.cc:5495] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66). New cstate: current_term: 1 leader_uuid: "bb0187aed98945ab8e104a9fdf3d4401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } } }
I20260810 00:03:27.445367 7664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:27.446157 7664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:27.480595 7502 heartbeater.cc:503] Master 127.5.100.124:40327 was elected leader, sending a full tablet report...
I20260810 00:03:38.228890 7703 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb0187aed98945ab8e104a9fdf3d4401)
I20260810 00:03:38.230320 7704 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb0187aed98945ab8e104a9fdf3d4401)
I20260810 00:03:38.230089 7703 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.230918 7704 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.234629 7704 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:38.234644 7703 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:38.259646 7703 raft_consensus.cc:482] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b713c5b20574640948496d076483447)
I20260810 00:03:38.260300 7703 raft_consensus.cc:504] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.262405 7703 leader_election.cc:288] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
W20260810 00:03:38.449095 7336 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.100.66:39541 timed out after 0.213s (SENT) [suppressed 10 similar messages]
W20260810 00:03:38.547876 7336 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.100.66:39541, user_credentials={real_user=slave}} blocked reactor thread for 98959.1us
I20260810 00:03:38.663568 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.668488 7561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.674166 7706 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb0187aed98945ab8e104a9fdf3d4401)
I20260810 00:03:38.662961 7709 raft_consensus.cc:482] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb0187aed98945ab8e104a9fdf3d4401)
I20260810 00:03:38.676625 7709 raft_consensus.cc:504] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.680238 7387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:38.681294 7559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.682789 7562 raft_consensus.cc:2454] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b0468e66084ff59c01fb5b78d3f627 in term 1.
I20260810 00:03:38.683153 7709 leader_election.cc:288] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:38.674719 7706 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.685375 7561 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b0468e66084ff59c01fb5b78d3f627 in term 1.
I20260810 00:03:38.687249 7387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:38.687984 7387 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b713c5b20574640948496d076483447 in term 1.
I20260810 00:03:38.687721 7707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 065a70ec01b04c339834bd751671ad23)
I20260810 00:03:38.689025 7707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:38.689354 7526 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, 52b0468e66084ff59c01fb5b78d3f627; no voters:
I20260810 00:03:38.691641 7560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.692648 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.688429 7706 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:38.694613 7471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.695698 7706 raft_consensus.cc:2781] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:03:38.696216 7706 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb0187aed98945ab8e104a9fdf3d4401)
I20260810 00:03:38.696676 7706 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:38.697249 7558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.699491 7557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.702512 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.703382 7707 leader_election.cc:288] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:38.703639 7471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.714584 7474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.714728 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23" is_pre_election: true
I20260810 00:03:38.838413 7473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84aabfe16f1d4823bcc38f6ff128dffd" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.820281 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "52b0468e66084ff59c01fb5b78d3f627" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.840600 7174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a165b9cb2d1422ca7693fc2287e0471" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "60e7224900864daba37bc2346bf79e94" is_pre_election: true
I20260810 00:03:38.842677 7523 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b713c5b20574640948496d076483447; no voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:38.850740 7077 leader_election.cc:302] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a165b9cb2d1422ca7693fc2287e0471; no voters: 065a70ec01b04c339834bd751671ad23, 60e7224900864daba37bc2346bf79e94
I20260810 00:03:38.851390 7712 raft_consensus.cc:2726] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 00:03:38.852290 7336 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.100.67:44659, user_credentials={real_user=slave}} blocked reactor thread for 45530.3us
W20260810 00:03:38.853639 7336 leader_election.cc:334] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541): Timed out: RequestConsensusVote RPC to 127.5.100.66:39541 timed out after 0.213s (SENT)
I20260810 00:03:38.854131 7336 leader_election.cc:302] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627; no voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:38.851162 7706 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.858392 7711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 065a70ec01b04c339834bd751671ad23)
I20260810 00:03:38.860149 7706 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 election: Requested vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541)
I20260810 00:03:38.859817 7711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } }
I20260810 00:03:38.862177 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627"
I20260810 00:03:38.863212 7711 leader_election.cc:288] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747), 065a70ec01b04c339834bd751671ad23 (127.5.100.124:40327)
I20260810 00:03:38.866513 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "1a165b9cb2d1422ca7693fc2287e0471" is_pre_election: true
I20260810 00:03:38.869913 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60e7224900864daba37bc2346bf79e94" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "065a70ec01b04c339834bd751671ad23" is_pre_election: true
W20260810 00:03:38.870024 7561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60244 (request call id 5) took 206 ms (client timeout 266 ms). Trace:
W20260810 00:03:38.872160 7561 rpcz_store.cc:269] 0810 00:03:38.663444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0810 00:03:38.668024 (+ 4580us) service_pool.cc:227] Handling call
0810 00:03:38.869867 (+201843us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5400410354. stack: 00007f56fc10d033 00007f56fc10cf30 00007f56fba6b534 00000000003b4464 00000000003b43f9 00007f5702931471 00007f57029b02ae 00007f5704bb9c27 00007f5702541c04 00007f5702541b92 00007f5702541ac1 00007f5702541a3c 00007f5702540cf2 00007f5700c6d94c 00007f5700c6cd86 00007f5700c6c72f
0810 00:03:38.869971 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":36866816}
I20260810 00:03:38.872721 7147 leader_election.cc:302] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60e7224900864daba37bc2346bf79e94; no voters: 065a70ec01b04c339834bd751671ad23, 1a165b9cb2d1422ca7693fc2287e0471
I20260810 00:03:38.874621 7562 raft_consensus.cc:1226] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 2 FOLLOWER]: Rejecting Update request from peer bb0187aed98945ab8e104a9fdf3d4401 for earlier term 1. Current term is 2. Ops: []
I20260810 00:03:38.874859 7713 raft_consensus.cc:482] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b713c5b20574640948496d076483447)
I20260810 00:03:38.875362 7713 raft_consensus.cc:504] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.877389 7713 leader_election.cc:288] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:38.878265 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:38.878475 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401" is_pre_election: true
I20260810 00:03:38.879407 7707 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 00:03:38.881393 7474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "2b713c5b20574640948496d076483447" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb0187aed98945ab8e104a9fdf3d4401"
I20260810 00:03:38.881691 7702 consensus_queue.cc:1046] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260810 00:03:38.882393 7711 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 00:03:38.882586 7523 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b713c5b20574640948496d076483447; no voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:38.883908 7709 raft_consensus.cc:2726] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 00:03:38.884689 7714 raft_consensus.cc:2726] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:03:38.885030 7336 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627; no voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:38.886212 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da63227906254ceb8195cd2b134a5fd6" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:38.886322 7336 leader_election.cc:302] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627; no voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401
I20260810 00:03:38.885871 7714 raft_consensus.cc:2726] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:03:38.887593 7714 raft_consensus.cc:2726] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:03:38.888543 7434 leader_election.cc:302] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb0187aed98945ab8e104a9fdf3d4401; no voters: 2b713c5b20574640948496d076483447, 52b0468e66084ff59c01fb5b78d3f627
I20260810 00:03:38.889369 7713 raft_consensus.cc:3032] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 00:03:38.889515 7715 raft_consensus.cc:2726] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:03:38.889748 7713 raft_consensus.cc:729] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Running, Role: LEADER
I20260810 00:03:38.890527 7713 consensus_queue.cc:260] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:38.891448 7713 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:03:39.063895 7713 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:39.064414 7713 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:39.066053 7713 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:39.066560 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627" is_pre_election: true
I20260810 00:03:39.066838 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447" is_pre_election: true
I20260810 00:03:39.067109 7388 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 1.
I20260810 00:03:39.067342 7562 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 2.
I20260810 00:03:39.068017 7436 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b0468e66084ff59c01fb5b78d3f627, bb0187aed98945ab8e104a9fdf3d4401; no voters:
I20260810 00:03:39.068759 7713 raft_consensus.cc:2781] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 00:03:39.069100 7713 raft_consensus.cc:482] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:39.069461 7713 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:39.075750 7713 raft_consensus.cc:504] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:39.077507 7713 leader_election.cc:288] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 3 election: Requested vote from peers 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289), 2b713c5b20574640948496d076483447 (127.5.100.67:44659)
I20260810 00:03:39.078249 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2b713c5b20574640948496d076483447"
I20260810 00:03:39.078562 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039aeddf930c46c6bb2230b9e95d229d" candidate_uuid: "bb0187aed98945ab8e104a9fdf3d4401" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52b0468e66084ff59c01fb5b78d3f627"
I20260810 00:03:39.078802 7562 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 2 FOLLOWER]: Advancing to term 3
I20260810 00:03:39.079098 7388 raft_consensus.cc:3037] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Advancing to term 3
I20260810 00:03:39.083621 7562 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 3.
I20260810 00:03:39.084033 7388 raft_consensus.cc:2454] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb0187aed98945ab8e104a9fdf3d4401 in term 3.
I20260810 00:03:39.084404 7434 leader_election.cc:302] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b713c5b20574640948496d076483447, bb0187aed98945ab8e104a9fdf3d4401; no voters:
I20260810 00:03:39.085090 7713 raft_consensus.cc:2781] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 00:03:39.085474 7713 raft_consensus.cc:686] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 3 LEADER]: Becoming Leader. State: Replica: bb0187aed98945ab8e104a9fdf3d4401, State: Running, Role: LEADER
I20260810 00:03:39.086184 7713 consensus_queue.cc:237] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } }
I20260810 00:03:39.096746 7236 catalog_manager.cc:5495] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bb0187aed98945ab8e104a9fdf3d4401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb0187aed98945ab8e104a9fdf3d4401" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 39541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:39.208256 7562 raft_consensus.cc:1261] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 3 FOLLOWER]: Refusing update from remote peer bb0187aed98945ab8e104a9fdf3d4401: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260810 00:03:39.209430 7713 consensus_queue.cc:1035] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b713c5b20574640948496d076483447" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 00:03:39.210186 7388 raft_consensus.cc:1261] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 3 FOLLOWER]: Refusing update from remote peer bb0187aed98945ab8e104a9fdf3d4401: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260810 00:03:39.211331 7713 consensus_queue.cc:1035] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b0468e66084ff59c01fb5b78d3f627" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260810 00:03:39.709461 7583 debug-util.cc:398] Leaking SignalData structure 0x7b080021ab00 after lost signal to thread 5524
W20260810 00:03:39.710444 7583 debug-util.cc:398] Leaking SignalData structure 0x7b080021b260 after lost signal to thread 7129
W20260810 00:03:39.711237 7583 debug-util.cc:398] Leaking SignalData structure 0x7b080021b920 after lost signal to thread 7200
W20260810 00:03:39.711920 7583 debug-util.cc:398] Leaking SignalData structure 0x7b0800113000 after lost signal to thread 7274
W20260810 00:03:39.712970 7583 debug-util.cc:398] Leaking SignalData structure 0x7b080023dae0 after lost signal to thread 7415
W20260810 00:03:39.713755 7583 debug-util.cc:398] Leaking SignalData structure 0x7b08002362c0 after lost signal to thread 7501
W20260810 00:03:39.714526 7583 debug-util.cc:398] Leaking SignalData structure 0x7b080013f100 after lost signal to thread 7586
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:783: Failure
Expected equality of these values:
live_row_count
Which is: 67
table_info->GetMetrics()->live_row_count->value()
Which is: 0
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_util.cc:395: Failure
Failed
Timed out waiting for assertion to pass.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:767: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:785: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 782, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:822: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260810 00:03:58.001287 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:58.041656 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:58.042307 5521 tablet_replica.cc:330] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627: stopping tablet replica
I20260810 00:03:58.042996 5521 raft_consensus.cc:2229] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.043563 5521 raft_consensus.cc:2258] T da63227906254ceb8195cd2b134a5fd6 P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.046144 5521 tablet_replica.cc:330] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627: stopping tablet replica
W20260810 00:03:58.046453 7436 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
I20260810 00:03:58.046687 5521 raft_consensus.cc:2229] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.047339 5521 raft_consensus.cc:2258] T 84aabfe16f1d4823bcc38f6ff128dffd P 52b0468e66084ff59c01fb5b78d3f627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.049610 5521 tablet_replica.cc:330] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627: stopping tablet replica
W20260810 00:03:58.049698 7436 consensus_peers.cc:487] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 -> Peer 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289): Couldn't send request to peer 52b0468e66084ff59c01fb5b78d3f627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:58.050114 5521 raft_consensus.cc:2229] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.050532 5521 raft_consensus.cc:2258] T 039aeddf930c46c6bb2230b9e95d229d P 52b0468e66084ff59c01fb5b78d3f627 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.074352 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
W20260810 00:03:58.078956 7526 consensus_peers.cc:487] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 -> Peer 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289): Couldn't send request to peer 52b0468e66084ff59c01fb5b78d3f627. Status: Network error: Client connection negotiation failed: client connection to 127.5.100.65:46289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:58.088248 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
W20260810 00:03:58.101722 7436 consensus_peers.cc:487] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 -> Peer 52b0468e66084ff59c01fb5b78d3f627 (127.5.100.65:46289): Couldn't send request to peer 52b0468e66084ff59c01fb5b78d3f627. Status: Network error: Client connection negotiation failed: client connection to 127.5.100.65:46289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:58.125509 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:58.126343 5521 tablet_replica.cc:330] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401: stopping tablet replica
I20260810 00:03:58.126906 5521 raft_consensus.cc:2229] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:58.127750 5521 raft_consensus.cc:2258] T 84aabfe16f1d4823bcc38f6ff128dffd P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.129721 5521 tablet_replica.cc:330] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401: stopping tablet replica
I20260810 00:03:58.130237 5521 raft_consensus.cc:2229] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 3 LEADER]: Raft consensus shutting down.
I20260810 00:03:58.131003 5521 raft_consensus.cc:2258] T 039aeddf930c46c6bb2230b9e95d229d P bb0187aed98945ab8e104a9fdf3d4401 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.132758 5521 tablet_replica.cc:330] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401: stopping tablet replica
I20260810 00:03:58.133229 5521 raft_consensus.cc:2229] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.133652 5521 raft_consensus.cc:2258] T da63227906254ceb8195cd2b134a5fd6 P bb0187aed98945ab8e104a9fdf3d4401 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 00:03:58.143991 7523 consensus_peers.cc:487] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 -> Peer bb0187aed98945ab8e104a9fdf3d4401 (127.5.100.66:39541): Couldn't send request to peer bb0187aed98945ab8e104a9fdf3d4401. Status: Network error: Client connection negotiation failed: client connection to 127.5.100.66:39541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:58.162539 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:58.177446 5521 tablet_server.cc:178] TabletServer@127.5.100.67:0 shutting down...
I20260810 00:03:58.209048 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:58.209720 5521 tablet_replica.cc:330] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447: stopping tablet replica
I20260810 00:03:58.210644 5521 raft_consensus.cc:2229] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.211360 5521 raft_consensus.cc:2258] T 039aeddf930c46c6bb2230b9e95d229d P 2b713c5b20574640948496d076483447 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.213413 5521 tablet_replica.cc:330] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447: stopping tablet replica
I20260810 00:03:58.213860 5521 raft_consensus.cc:2229] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:58.214617 5521 raft_consensus.cc:2258] T da63227906254ceb8195cd2b134a5fd6 P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.216274 5521 tablet_replica.cc:330] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447: stopping tablet replica
I20260810 00:03:58.216684 5521 raft_consensus.cc:2229] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.217134 5521 raft_consensus.cc:2258] T 84aabfe16f1d4823bcc38f6ff128dffd P 2b713c5b20574640948496d076483447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.257886 5521 tablet_server.cc:194] TabletServer@127.5.100.67:0 shutdown complete.
I20260810 00:03:58.274188 5521 master.cc:537] Master@127.5.100.126:33747 shutting down...
I20260810 00:03:58.299685 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.300331 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.300683 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1a165b9cb2d1422ca7693fc2287e0471: stopping tablet replica
I20260810 00:03:58.319664 5521 master.cc:552] Master@127.5.100.126:33747 shutdown complete.
I20260810 00:03:58.336094 5521 master.cc:537] Master@127.5.100.125:42269 shutting down...
W20260810 00:03:58.350013 7219 consensus_peers.cc:487] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 -> Peer 1a165b9cb2d1422ca7693fc2287e0471 (127.5.100.126:33747): Couldn't send request to peer 1a165b9cb2d1422ca7693fc2287e0471. Status: Network error: Client connection negotiation failed: client connection to 127.5.100.126:33747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 00:03:58.361784 7216 consensus_peers.cc:487] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 -> Peer 60e7224900864daba37bc2346bf79e94 (127.5.100.125:42269): Couldn't send request to peer 60e7224900864daba37bc2346bf79e94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:03:58.362205 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:58.362834 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.363272 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 60e7224900864daba37bc2346bf79e94: stopping tablet replica
I20260810 00:03:58.402486 5521 master.cc:552] Master@127.5.100.125:42269 shutdown complete.
I20260810 00:03:58.420907 5521 master.cc:537] Master@127.5.100.124:40327 shutting down...
I20260810 00:03:58.441928 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 3 LEADER]: Raft consensus shutting down.
I20260810 00:03:58.441582 7771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "065a70ec01b04c339834bd751671ad23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a165b9cb2d1422ca7693fc2287e0471" member_type: VOTER last_known_addr { host: "127.5.100.126" port: 33747 } } peers { permanent_uuid: "60e7224900864daba37bc2346bf79e94" member_type: VOTER last_known_addr { host: "127.5.100.125" port: 42269 } } peers { permanent_uuid: "065a70ec01b04c339834bd751671ad23" member_type: VOTER last_known_addr { host: "127.5.100.124" port: 40327 } } }
I20260810 00:03:58.442566 7771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:58.442834 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:58.448755 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 065a70ec01b04c339834bd751671ad23: stopping tablet replica
I20260810 00:03:58.468489 5521 master.cc:552] Master@127.5.100.124:40327 shutdown complete.
I20260810 00:03:58.495923 5521 test_util.cc:175] -----------------------------------------------
I20260810 00:03:58.496212 5521 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786320191480876-5521-0
[ FAILED ] TsTabletManagerITest.TestTableStats (33124 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260810 00:03:58.502547 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:44459
I20260810 00:03:58.503672 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:58.509461 7790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.510993 7791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.511241 7793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.511945 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:58.512895 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:58.513100 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:58.513253 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320238513240 us; error 0 us; skew 500 ppm
I20260810 00:03:58.513780 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:58.519894 5521 webserver.cc:460] Webserver started at http://127.5.100.126:44033/ using document root <none> and password file <none>
I20260810 00:03:58.520458 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:58.520669 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:58.520977 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:58.522294 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "a8339d6ae63f4ccfb114201232959c17"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.526903 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:03:58.530171 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.531011 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:03:58.531246 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "a8339d6ae63f4ccfb114201232959c17"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.531490 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:58.541512 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:58.542538 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:58.579038 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:44459
I20260810 00:03:58.579149 7855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:44459 every 8 connection(s)
I20260810 00:03:58.582911 7856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:58.592602 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: Bootstrap starting.
I20260810 00:03:58.596926 7856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:58.601111 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: No bootstrap required, opened a new log
I20260810 00:03:58.603125 7856 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER }
I20260810 00:03:58.603523 7856 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:58.603751 7856 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8339d6ae63f4ccfb114201232959c17, State: Initialized, Role: FOLLOWER
I20260810 00:03:58.604359 7856 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER }
I20260810 00:03:58.604789 7856 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:58.605031 7856 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:58.605304 7856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:58.609275 7856 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER }
I20260810 00:03:58.609786 7856 leader_election.cc:302] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8339d6ae63f4ccfb114201232959c17; no voters:
I20260810 00:03:58.610842 7856 leader_election.cc:288] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:58.611143 7859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:58.612573 7859 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 1 LEADER]: Becoming Leader. State: Replica: a8339d6ae63f4ccfb114201232959c17, State: Running, Role: LEADER
I20260810 00:03:58.613222 7859 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER }
I20260810 00:03:58.613899 7856 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:58.618263 7860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8339d6ae63f4ccfb114201232959c17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER } }
I20260810 00:03:58.618528 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8339d6ae63f4ccfb114201232959c17. Latest consensus state: current_term: 1 leader_uuid: "a8339d6ae63f4ccfb114201232959c17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8339d6ae63f4ccfb114201232959c17" member_type: VOTER } }
I20260810 00:03:58.619093 7860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:58.619261 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:58.621896 7867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:58.626587 7867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:58.630657 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:58.635687 7867 catalog_manager.cc:1347] Generated new cluster ID: baa8ecea040f4f3494c60bdd796982b0
I20260810 00:03:58.635905 7867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:58.650708 7867 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:58.652205 7867 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:58.661322 7867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: Generated new TSK 0
I20260810 00:03:58.661813 7867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:58.697250 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:58.702978 7879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.704321 7880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.706211 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:58.706670 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.707716 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:58.707892 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:58.708004 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320238707996 us; error 0 us; skew 500 ppm
I20260810 00:03:58.708566 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:58.710737 5521 webserver.cc:460] Webserver started at http://127.5.100.65:44399/ using document root <none> and password file <none>
I20260810 00:03:58.711151 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:58.711301 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:58.711547 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:58.712621 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "2a54bafc65f347c790dd38a63d07257d"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.717046 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 00:03:58.719992 7888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.720744 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:03:58.721025 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "2a54bafc65f347c790dd38a63d07257d"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.721319 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:58.741029 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:58.742055 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:58.743423 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:58.745647 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:58.745857 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.746105 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:58.746276 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.785311 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:37647
I20260810 00:03:58.785406 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:37647 every 8 connection(s)
I20260810 00:03:58.790314 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:58.797385 7962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.798632 7963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.802629 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:58.803592 7965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.804602 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:58.804886 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:58.805071 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320238805056 us; error 0 us; skew 500 ppm
I20260810 00:03:58.805522 7958 heartbeater.cc:347] Connected to a master server at 127.5.100.126:44459
I20260810 00:03:58.805768 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:58.805935 7958 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:58.806831 7958 heartbeater.cc:511] Master 127.5.100.126:44459 requested a full tablet report, sending...
I20260810 00:03:58.808503 5521 webserver.cc:460] Webserver started at http://127.5.100.66:34261/ using document root <none> and password file <none>
I20260810 00:03:58.808987 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:58.809010 7817 ts_manager.cc:194] Registered new tserver with Master: 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647)
I20260810 00:03:58.809268 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:58.809597 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:58.810721 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "3f68cf9cd5ce49f8aa52a1621afa8733"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.811074 7817 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45770
I20260810 00:03:58.815706 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:03:58.819267 7972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.819979 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:03:58.820327 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "3f68cf9cd5ce49f8aa52a1621afa8733"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.820710 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:58.841109 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:58.842139 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:58.843549 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:58.845666 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:58.845870 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.846115 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:58.846282 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.884864 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:45263
I20260810 00:03:58.884976 8040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:45263 every 8 connection(s)
I20260810 00:03:58.889560 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:58.897576 8045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.897831 8044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:58.902207 8047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:58.904320 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:58.905324 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:58.905572 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:58.905789 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320238905759 us; error 0 us; skew 500 ppm
I20260810 00:03:58.905961 8041 heartbeater.cc:347] Connected to a master server at 127.5.100.126:44459
I20260810 00:03:58.906348 8041 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:58.906422 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:58.907195 8041 heartbeater.cc:511] Master 127.5.100.126:44459 requested a full tablet report, sending...
I20260810 00:03:58.909058 5521 webserver.cc:460] Webserver started at http://127.5.100.67:35395/ using document root <none> and password file <none>
I20260810 00:03:58.909386 7817 ts_manager.cc:194] Registered new tserver with Master: 3f68cf9cd5ce49f8aa52a1621afa8733 (127.5.100.66:45263)
I20260810 00:03:58.909719 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:58.909992 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:58.910293 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:58.911543 7817 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45780
I20260810 00:03:58.911782 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root/instance:
uuid: "b1af48e9fc27416b85f3aa1a4e245fc2"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.916440 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:03:58.919358 8052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.920076 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 00:03:58.920390 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root
uuid: "b1af48e9fc27416b85f3aa1a4e245fc2"
format_stamp: "Formatted at 2026-08-10 00:03:58 on dist-test-slave-8w9v"
I20260810 00:03:58.920620 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:58.941558 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:58.942680 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:58.944046 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:03:58.946281 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:58.946493 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.946765 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:58.946928 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:58.986784 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.67:33021
I20260810 00:03:58.986876 8120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.67:33021 every 8 connection(s)
I20260810 00:03:59.004084 8121 heartbeater.cc:347] Connected to a master server at 127.5.100.126:44459
I20260810 00:03:59.004514 8121 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:59.005235 8121 heartbeater.cc:511] Master 127.5.100.126:44459 requested a full tablet report, sending...
I20260810 00:03:59.007048 7817 ts_manager.cc:194] Registered new tserver with Master: b1af48e9fc27416b85f3aa1a4e245fc2 (127.5.100.67:33021)
I20260810 00:03:59.007371 5521 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017298995s
I20260810 00:03:59.009236 7817 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45796
I20260810 00:03:59.044765 7817 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260810 00:03:59.046404 7817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 00:03:59.093147 8002 tablet_service.cc:1467] Processing CreateTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 (DEFAULT_TABLE table=test-table [id=d046ab9d9b064c97add17e6a477a127b]), partition=
I20260810 00:03:59.094744 8002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efd989b9d9f4a08a464d7343fc16400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:59.100775 7921 tablet_service.cc:1467] Processing CreateTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 (DEFAULT_TABLE table=test-table [id=d046ab9d9b064c97add17e6a477a127b]), partition=
I20260810 00:03:59.101755 7921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efd989b9d9f4a08a464d7343fc16400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:59.103762 8083 tablet_service.cc:1467] Processing CreateTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 (DEFAULT_TABLE table=test-table [id=d046ab9d9b064c97add17e6a477a127b]), partition=
I20260810 00:03:59.104864 8083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efd989b9d9f4a08a464d7343fc16400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:59.109524 8139 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Bootstrap starting.
I20260810 00:03:59.114941 8139 tablet_bootstrap.cc:654] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:59.117657 8141 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Bootstrap starting.
I20260810 00:03:59.120417 8139 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: No bootstrap required, opened a new log
I20260810 00:03:59.120867 8143 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Bootstrap starting.
I20260810 00:03:59.120955 8139 ts_tablet_manager.cc:1397] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260810 00:03:59.124198 8139 raft_consensus.cc:348] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.125005 8139 raft_consensus.cc:374] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:59.125315 8139 raft_consensus.cc:729] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f68cf9cd5ce49f8aa52a1621afa8733, State: Initialized, Role: FOLLOWER
I20260810 00:03:59.126590 8143 tablet_bootstrap.cc:654] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:59.127243 8141 tablet_bootstrap.cc:654] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:59.126041 8139 consensus_queue.cc:260] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.131201 8143 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: No bootstrap required, opened a new log
I20260810 00:03:59.131368 8139 ts_tablet_manager.cc:1428] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
W20260810 00:03:59.131932 8139 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260810 00:03:59.132031 8143 ts_tablet_manager.cc:1397] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260810 00:03:59.131204 8041 heartbeater.cc:503] Master 127.5.100.126:44459 was elected leader, sending a full tablet report...
I20260810 00:03:59.134698 8143 raft_consensus.cc:348] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.135351 8143 raft_consensus.cc:374] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:59.135674 8143 raft_consensus.cc:729] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1af48e9fc27416b85f3aa1a4e245fc2, State: Initialized, Role: FOLLOWER
I20260810 00:03:59.136510 8143 consensus_queue.cc:260] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.137701 8141 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: No bootstrap required, opened a new log
I20260810 00:03:59.138197 8141 ts_tablet_manager.cc:1397] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.004s
W20260810 00:03:59.138648 8042 tablet.cc:2369] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 00:03:59.138906 8121 heartbeater.cc:503] Master 127.5.100.126:44459 was elected leader, sending a full tablet report...
I20260810 00:03:59.139174 8143 ts_tablet_manager.cc:1428] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
W20260810 00:03:59.139458 8143 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260810 00:03:59.140942 8141 raft_consensus.cc:348] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.141583 8141 raft_consensus.cc:374] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:59.141822 8141 raft_consensus.cc:729] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a54bafc65f347c790dd38a63d07257d, State: Initialized, Role: FOLLOWER
I20260810 00:03:59.142411 8141 consensus_queue.cc:260] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.144624 7958 heartbeater.cc:503] Master 127.5.100.126:44459 was elected leader, sending a full tablet report...
I20260810 00:03:59.145036 8141 ts_tablet_manager.cc:1428] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
W20260810 00:03:59.145279 8141 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260810 00:03:59.202888 8147 raft_consensus.cc:482] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:03:59.203437 8147 raft_consensus.cc:504] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.205610 8147 leader_election.cc:288] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f68cf9cd5ce49f8aa52a1621afa8733 (127.5.100.66:45263), 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647)
I20260810 00:03:59.221805 7932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0efd989b9d9f4a08a464d7343fc16400" candidate_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a54bafc65f347c790dd38a63d07257d" is_pre_election: true
I20260810 00:03:59.222402 8013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0efd989b9d9f4a08a464d7343fc16400" candidate_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" is_pre_election: true
I20260810 00:03:59.222684 7932 raft_consensus.cc:2454] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1af48e9fc27416b85f3aa1a4e245fc2 in term 0.
I20260810 00:03:59.223078 8013 raft_consensus.cc:2454] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1af48e9fc27416b85f3aa1a4e245fc2 in term 0.
I20260810 00:03:59.224028 8057 leader_election.cc:302] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f68cf9cd5ce49f8aa52a1621afa8733, b1af48e9fc27416b85f3aa1a4e245fc2; no voters:
I20260810 00:03:59.224753 8147 raft_consensus.cc:2781] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:03:59.225045 8147 raft_consensus.cc:482] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:03:59.225250 8147 raft_consensus.cc:3037] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:59.229529 8147 raft_consensus.cc:504] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.231207 8147 leader_election.cc:288] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [CANDIDATE]: Term 1 election: Requested vote from peers 3f68cf9cd5ce49f8aa52a1621afa8733 (127.5.100.66:45263), 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647)
I20260810 00:03:59.231942 8013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0efd989b9d9f4a08a464d7343fc16400" candidate_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733"
I20260810 00:03:59.232090 7932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0efd989b9d9f4a08a464d7343fc16400" candidate_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a54bafc65f347c790dd38a63d07257d"
I20260810 00:03:59.232478 8013 raft_consensus.cc:3037] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:59.232677 7932 raft_consensus.cc:3037] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:59.237042 8013 raft_consensus.cc:2454] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1af48e9fc27416b85f3aa1a4e245fc2 in term 1.
I20260810 00:03:59.237223 7932 raft_consensus.cc:2454] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1af48e9fc27416b85f3aa1a4e245fc2 in term 1.
I20260810 00:03:59.237954 8057 leader_election.cc:302] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f68cf9cd5ce49f8aa52a1621afa8733, b1af48e9fc27416b85f3aa1a4e245fc2; no voters:
I20260810 00:03:59.238623 8147 raft_consensus.cc:2781] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:59.239424 8147 raft_consensus.cc:686] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 LEADER]: Becoming Leader. State: Replica: b1af48e9fc27416b85f3aa1a4e245fc2, State: Running, Role: LEADER
I20260810 00:03:59.240180 8147 consensus_queue.cc:237] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.247714 7817 catalog_manager.cc:5495] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1af48e9fc27416b85f3aa1a4e245fc2 (127.5.100.67). New cstate: current_term: 1 leader_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } health_report { overall_health: UNKNOWN } } }
I20260810 00:03:59.323081 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:49640
I20260810 00:03:59.325083 8154 tablet_replica.cc:330] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: stopping tablet replica
I20260810 00:03:59.325798 8154 raft_consensus.cc:2229] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:59.326313 8154 raft_consensus.cc:2258] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:59.328596 8154 ts_tablet_manager.cc:1905] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260810 00:03:59.339550 8154 ts_tablet_manager.cc:1918] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260810 00:03:59.339957 8154 log.cc:1196] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root/wals/0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.343992 8155 ts_tablet_manager.cc:927] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Initiating tablet copy from peer b1af48e9fc27416b85f3aa1a4e245fc2 (127.5.100.67:33021)
I20260810 00:03:59.345525 8155 tablet_copy_client.cc:316] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet copy: Beginning tablet copy session from remote peer at address 127.5.100.67:33021
I20260810 00:03:59.356356 8104 tablet_copy_service.cc:140] P b1af48e9fc27416b85f3aa1a4e245fc2: Received BeginTabletCopySession request for tablet 0efd989b9d9f4a08a464d7343fc16400 from peer 2a54bafc65f347c790dd38a63d07257d ({username='slave'} at 127.0.0.1:50388)
I20260810 00:03:59.356765 8104 tablet_copy_service.cc:161] P b1af48e9fc27416b85f3aa1a4e245fc2: Beginning new tablet copy session on tablet 0efd989b9d9f4a08a464d7343fc16400 from peer 2a54bafc65f347c790dd38a63d07257d at {username='slave'} at 127.0.0.1:50388: session id = 2a54bafc65f347c790dd38a63d07257d-0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.361851 8104 tablet_copy_source_session.cc:205] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:03:59.363963 8155 ts_tablet_manager.cc:1905] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260810 00:03:59.372681 8155 ts_tablet_manager.cc:1918] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260810 00:03:59.373203 8155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efd989b9d9f4a08a464d7343fc16400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:59.377743 8155 tablet_copy_client.cc:791] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet copy: Starting download of 0 data blocks...
I20260810 00:03:59.378258 8155 tablet_copy_client.cc:655] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet copy: Starting download of 1 WAL segments...
I20260810 00:03:59.379833 7817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45800:
table { table_id: "d046ab9d9b064c97add17e6a477a127b" table_name: "test-table" }
I20260810 00:03:59.380465 7817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45800:
table { table_id: "d046ab9d9b064c97add17e6a477a127b" table_name: "test-table" }
I20260810 00:03:59.382227 8155 tablet_copy_client.cc:531] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:03:59.387543 8155 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Bootstrap starting.
I20260810 00:03:59.389626 7817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: Sending DeleteTablet for 3 replicas of tablet 0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.391544 8083 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:03:59 UTC) from {username='slave'} at 127.0.0.1:50380
I20260810 00:03:59.392022 8002 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:03:59 UTC) from {username='slave'} at 127.0.0.1:44106
I20260810 00:03:59.393465 8158 tablet_replica.cc:330] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: stopping tablet replica
I20260810 00:03:59.394110 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:03:59 UTC) from {username='slave'} at 127.0.0.1:49622
I20260810 00:03:59.395445 8159 tablet_replica.cc:330] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: stopping tablet replica
I20260810 00:03:59.395499 8158 raft_consensus.cc:2229] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:59.396406 8159 raft_consensus.cc:2229] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:59.396452 8158 pending_rounds.cc:62] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Trying to abort 1 pending ops.
I20260810 00:03:59.397094 8159 raft_consensus.cc:2258] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:59.397048 8158 pending_rounds.cc:68] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316767699934560256 op_type: NO_OP noop_request { }
I20260810 00:03:59.397809 8158 raft_consensus.cc:2866] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 00:03:59.398258 8158 raft_consensus.cc:2258] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 00:03:59.398466 7804 catalog_manager.cc:4821] TS 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647): delete failed for tablet 0efd989b9d9f4a08a464d7343fc16400 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0efd989b9d9f4a08a464d7343fc16400 already in progress: copying tablet
I20260810 00:03:59.400080 8159 ts_tablet_manager.cc:1905] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:03:59.401839 8158 ts_tablet_manager.cc:1905] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:03:59.404186 8155 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260810 00:03:59.405128 8155 tablet_bootstrap.cc:492] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Bootstrap complete.
I20260810 00:03:59.405752 8155 ts_tablet_manager.cc:1397] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.010s
I20260810 00:03:59.408463 8155 raft_consensus.cc:348] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.409415 8155 raft_consensus.cc:729] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a54bafc65f347c790dd38a63d07257d, State: Initialized, Role: FOLLOWER
I20260810 00:03:59.410158 8155 consensus_queue.cc:260] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af48e9fc27416b85f3aa1a4e245fc2" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 33021 } } peers { permanent_uuid: "3f68cf9cd5ce49f8aa52a1621afa8733" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 45263 } } peers { permanent_uuid: "2a54bafc65f347c790dd38a63d07257d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 37647 } }
I20260810 00:03:59.412184 8155 ts_tablet_manager.cc:1428] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260810 00:03:59.412420 8155 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260810 00:03:59.413021 8158 ts_tablet_manager.cc:1918] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 00:03:59.413353 8158 log.cc:1196] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-2-root/wals/0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.414023 8158 ts_tablet_manager.cc:1939] T 0efd989b9d9f4a08a464d7343fc16400 P b1af48e9fc27416b85f3aa1a4e245fc2: Deleting consensus metadata
I20260810 00:03:59.414800 8159 ts_tablet_manager.cc:1918] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 00:03:59.415119 8159 log.cc:1196] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-1-root/wals/0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.415426 7803 catalog_manager.cc:4841] TS b1af48e9fc27416b85f3aa1a4e245fc2 (127.5.100.67:33021): tablet 0efd989b9d9f4a08a464d7343fc16400 (table test-table [id=d046ab9d9b064c97add17e6a477a127b]) successfully deleted
I20260810 00:03:59.415989 8159 ts_tablet_manager.cc:1939] T 0efd989b9d9f4a08a464d7343fc16400 P 3f68cf9cd5ce49f8aa52a1621afa8733: Deleting consensus metadata
I20260810 00:03:59.417694 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:03:59 UTC) from {username='slave'} at 127.0.0.1:49622
I20260810 00:03:59.418655 7804 catalog_manager.cc:4841] TS 3f68cf9cd5ce49f8aa52a1621afa8733 (127.5.100.66:45263): tablet 0efd989b9d9f4a08a464d7343fc16400 (table test-table [id=d046ab9d9b064c97add17e6a477a127b]) successfully deleted
W20260810 00:03:59.419309 7804 catalog_manager.cc:4821] TS 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647): delete failed for tablet 0efd989b9d9f4a08a464d7343fc16400 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0efd989b9d9f4a08a464d7343fc16400 already in progress: copying tablet
I20260810 00:03:59.464286 8104 tablet_copy_service.cc:342] P b1af48e9fc27416b85f3aa1a4e245fc2: Request end of tablet copy session 2a54bafc65f347c790dd38a63d07257d-0efd989b9d9f4a08a464d7343fc16400 received from {username='slave'} at 127.0.0.1:50388
I20260810 00:03:59.464704 8104 tablet_copy_service.cc:433] P b1af48e9fc27416b85f3aa1a4e245fc2: ending tablet copy session 2a54bafc65f347c790dd38a63d07257d-0efd989b9d9f4a08a464d7343fc16400 on tablet 0efd989b9d9f4a08a464d7343fc16400 with peer 2a54bafc65f347c790dd38a63d07257d
I20260810 00:03:59.468849 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 0efd989b9d9f4a08a464d7343fc16400 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 00:03:59 UTC) from {username='slave'} at 127.0.0.1:49622
I20260810 00:03:59.469543 8154 tablet_replica.cc:330] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: stopping tablet replica
I20260810 00:03:59.470106 8154 raft_consensus.cc:2229] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:03:59.470482 8154 pending_rounds.cc:62] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Trying to abort 1 pending ops.
I20260810 00:03:59.470651 8154 pending_rounds.cc:68] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316767699934560256 op_type: NO_OP noop_request { }
I20260810 00:03:59.470952 8154 raft_consensus.cc:2866] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260810 00:03:59.471235 8154 raft_consensus.cc:2258] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:59.473196 8154 ts_tablet_manager.cc:1905] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 00:03:59.481824 8154 ts_tablet_manager.cc:1918] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 00:03:59.482088 8154 log.cc:1196] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786320191480876-5521-0/minicluster-data/ts-0-root/wals/0efd989b9d9f4a08a464d7343fc16400
I20260810 00:03:59.482717 8154 ts_tablet_manager.cc:1939] T 0efd989b9d9f4a08a464d7343fc16400 P 2a54bafc65f347c790dd38a63d07257d: Deleting consensus metadata
I20260810 00:03:59.484607 7804 catalog_manager.cc:4841] TS 2a54bafc65f347c790dd38a63d07257d (127.5.100.65:37647): tablet 0efd989b9d9f4a08a464d7343fc16400 (table test-table [id=d046ab9d9b064c97add17e6a477a127b]) successfully deleted
I20260810 00:03:59.533679 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:03:59.550385 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:59.567176 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:03:59.575899 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:03:59.591714 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:59.608395 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:03:59.617055 5521 tablet_server.cc:178] TabletServer@127.5.100.67:0 shutting down...
I20260810 00:03:59.632426 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:03:59.639241 5521 tablet_server.cc:194] TabletServer@127.5.100.67:0 shutdown complete.
I20260810 00:03:59.647468 5521 master.cc:537] Master@127.5.100.126:44459 shutting down...
I20260810 00:03:59.662259 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:03:59.662765 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:03:59.663055 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P a8339d6ae63f4ccfb114201232959c17: stopping tablet replica
I20260810 00:03:59.671056 5521 master.cc:552] Master@127.5.100.126:44459 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1191 ms)
[----------] 4 tests from TsTabletManagerITest (37430 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260810 00:03:59.694051 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:37959
I20260810 00:03:59.694958 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:59.699556 8165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:59.700788 8166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:59.701934 5521 server_base.cc:1034] running on GCE node
W20260810 00:03:59.702140 8169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:59.703019 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:59.703212 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:59.703397 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320239703384 us; error 0 us; skew 500 ppm
I20260810 00:03:59.703892 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:59.706023 5521 webserver.cc:460] Webserver started at http://127.5.100.126:45563/ using document root <none> and password file <none>
I20260810 00:03:59.706482 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:59.706666 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:59.706923 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:59.708014 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "d2f0d84650c34ef8a1ec00535a9e816d"
format_stamp: "Formatted at 2026-08-10 00:03:59 on dist-test-slave-8w9v"
I20260810 00:03:59.712165 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 00:03:59.715221 8173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:59.716070 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:03:59.716367 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "d2f0d84650c34ef8a1ec00535a9e816d"
format_stamp: "Formatted at 2026-08-10 00:03:59 on dist-test-slave-8w9v"
I20260810 00:03:59.716602 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:59.739656 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:59.740617 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:59.775416 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:37959
I20260810 00:03:59.775535 8229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:37959 every 8 connection(s)
I20260810 00:03:59.779350 8230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:03:59.788702 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d: Bootstrap starting.
I20260810 00:03:59.792850 8230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d: Neither blocks nor log segments found. Creating new log.
I20260810 00:03:59.796331 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d: No bootstrap required, opened a new log
I20260810 00:03:59.798118 8230 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER }
I20260810 00:03:59.798478 8230 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:03:59.798652 8230 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f0d84650c34ef8a1ec00535a9e816d, State: Initialized, Role: FOLLOWER
I20260810 00:03:59.799178 8230 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER }
I20260810 00:03:59.799566 8230 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:03:59.799749 8230 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:03:59.799962 8230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:03:59.803534 8230 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER }
I20260810 00:03:59.803932 8230 leader_election.cc:302] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f0d84650c34ef8a1ec00535a9e816d; no voters:
I20260810 00:03:59.804941 8230 leader_election.cc:288] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:03:59.805284 8235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:03:59.806689 8235 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 1 LEADER]: Becoming Leader. State: Replica: d2f0d84650c34ef8a1ec00535a9e816d, State: Running, Role: LEADER
I20260810 00:03:59.807328 8235 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER }
I20260810 00:03:59.807832 8230 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:03:59.810340 8238 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f0d84650c34ef8a1ec00535a9e816d. Latest consensus state: current_term: 1 leader_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER } }
I20260810 00:03:59.810256 8236 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f0d84650c34ef8a1ec00535a9e816d" member_type: VOTER } }
I20260810 00:03:59.810999 8238 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:59.811332 8236 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [sys.catalog]: This master's current role is: LEADER
I20260810 00:03:59.813642 8241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:03:59.821398 8241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:03:59.824450 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:03:59.830394 8241 catalog_manager.cc:1347] Generated new cluster ID: 5dd215d25b264190a9c0cd4265e10258
I20260810 00:03:59.830660 8241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:03:59.845229 8241 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:03:59.846768 8241 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:03:59.862486 8241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d: Generated new TSK 0
I20260810 00:03:59.863037 8241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:03:59.891392 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:03:59.897277 8255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:59.898428 8256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:03:59.899855 8258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:03:59.900213 5521 server_base.cc:1034] running on GCE node
I20260810 00:03:59.900972 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:03:59.901160 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:03:59.901322 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320239901308 us; error 0 us; skew 500 ppm
I20260810 00:03:59.901809 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:03:59.904059 5521 webserver.cc:460] Webserver started at http://127.5.100.65:36835/ using document root <none> and password file <none>
I20260810 00:03:59.904587 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:03:59.904772 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:03:59.905023 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:03:59.906036 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "13e717dcc6674905ae00b5c37394e840"
format_stamp: "Formatted at 2026-08-10 00:03:59 on dist-test-slave-8w9v"
I20260810 00:03:59.910802 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:03:59.913890 8264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:59.914691 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:03:59.914970 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "13e717dcc6674905ae00b5c37394e840"
format_stamp: "Formatted at 2026-08-10 00:03:59 on dist-test-slave-8w9v"
I20260810 00:03:59.915225 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:03:59.926046 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:03:59.926994 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:03:59.931643 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:03:59.931972 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:59.932349 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:03:59.932623 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:03:59.980988 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:46459
I20260810 00:03:59.981071 8339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:46459 every 8 connection(s)
I20260810 00:03:59.985193 8340 heartbeater.cc:347] Connected to a master server at 127.5.100.126:37959
I20260810 00:03:59.985544 8340 heartbeater.cc:464] Registering TS with master...
I20260810 00:03:59.986199 8340 heartbeater.cc:511] Master 127.5.100.126:37959 requested a full tablet report, sending...
I20260810 00:03:59.988128 8191 ts_manager.cc:194] Registered new tserver with Master: 13e717dcc6674905ae00b5c37394e840 (127.5.100.65:46459)
I20260810 00:03:59.988740 5521 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004793732s
I20260810 00:03:59.989969 8191 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53606
I20260810 00:04:00.023659 8303 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260810 00:04:00.025352 8303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:00.036931 8352 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Bootstrap starting.
I20260810 00:04:00.041056 8352 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:00.044822 8352 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: No bootstrap required, opened a new log
I20260810 00:04:00.045123 8352 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 00:04:00.046854 8352 raft_consensus.cc:348] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.047222 8352 raft_consensus.cc:374] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:00.047391 8352 raft_consensus.cc:729] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e717dcc6674905ae00b5c37394e840, State: Initialized, Role: FOLLOWER
I20260810 00:04:00.047843 8352 consensus_queue.cc:260] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.048331 8352 raft_consensus.cc:388] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:00.048513 8352 raft_consensus.cc:482] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:00.048699 8352 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:00.052824 8352 raft_consensus.cc:504] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.053270 8352 leader_election.cc:302] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13e717dcc6674905ae00b5c37394e840; no voters:
I20260810 00:04:00.054266 8352 leader_election.cc:288] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:00.054600 8355 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:00.056265 8340 heartbeater.cc:503] Master 127.5.100.126:37959 was elected leader, sending a full tablet report...
I20260810 00:04:00.056463 8352 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 00:04:00.056828 8355 raft_consensus.cc:686] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 LEADER]: Becoming Leader. State: Replica: 13e717dcc6674905ae00b5c37394e840, State: Running, Role: LEADER
I20260810 00:04:00.057453 8355 consensus_queue.cc:237] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.061115 8356 tablet_replica.cc:439] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13e717dcc6674905ae00b5c37394e840" committed_config { opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } } }
I20260810 00:04:00.061836 8356 tablet_replica.cc:442] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:00.063171 8357 tablet_replica.cc:439] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: TxnStatusTablet state changed. Reason: New leader 13e717dcc6674905ae00b5c37394e840. Latest consensus state: current_term: 1 leader_uuid: "13e717dcc6674905ae00b5c37394e840" committed_config { opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } } }
I20260810 00:04:00.063694 8357 tablet_replica.cc:442] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:00.063933 8360 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:00.064508 8360 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260810 00:04:00.064848 8191 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260810 00:04:00.069630 8191 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:00.100639 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:00.120643 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:00.121377 5521 tablet_replica.cc:330] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: stopping tablet replica
I20260810 00:04:00.122062 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:00.122496 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:00.143567 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:00.153613 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:00.158954 8363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.159102 8364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:00.161849 5521 server_base.cc:1034] running on GCE node
W20260810 00:04:00.161808 8366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:00.163128 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:00.163337 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:00.163498 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320240163485 us; error 0 us; skew 500 ppm
I20260810 00:04:00.164072 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:00.166923 5521 webserver.cc:460] Webserver started at http://127.5.100.65:36835/ using document root <none> and password file <none>
I20260810 00:04:00.167394 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:00.167588 5521 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260810 00:04:00.171339 5521 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 00:04:00.174185 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.174944 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 00:04:00.175213 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "13e717dcc6674905ae00b5c37394e840"
format_stamp: "Formatted at 2026-08-10 00:03:59 on dist-test-slave-8w9v"
I20260810 00:04:00.175508 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:00.196223 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:00.197160 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:00.205199 8382 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 00:04:00.214040 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 00:04:00.214327 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.002s sys 0.000s
I20260810 00:04:00.214651 5521 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 00:04:00.221169 5521 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 00:04:00.221411 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.006s sys 0.000s
I20260810 00:04:00.223577 8382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Bootstrap starting.
I20260810 00:04:00.264917 8382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:04:00.266072 8382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Bootstrap complete.
I20260810 00:04:00.266846 8382 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Time spent bootstrapping tablet: real 0.043s user 0.033s sys 0.003s
I20260810 00:04:00.269878 8382 raft_consensus.cc:348] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.270503 8382 raft_consensus.cc:729] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e717dcc6674905ae00b5c37394e840, State: Initialized, Role: FOLLOWER
I20260810 00:04:00.271489 8382 consensus_queue.cc:260] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.272324 8382 raft_consensus.cc:388] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:00.272652 8382 raft_consensus.cc:482] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:00.273051 8382 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:04:00.281692 8382 raft_consensus.cc:504] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.282560 8382 leader_election.cc:302] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13e717dcc6674905ae00b5c37394e840; no voters:
I20260810 00:04:00.284281 8382 leader_election.cc:288] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:04:00.284593 8418 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:04:00.293411 8382 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840: Time spent starting tablet: real 0.026s user 0.017s sys 0.010s
I20260810 00:04:00.294677 8418 raft_consensus.cc:686] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 2 LEADER]: Becoming Leader. State: Replica: 13e717dcc6674905ae00b5c37394e840, State: Running, Role: LEADER
I20260810 00:04:00.296520 8418 consensus_queue.cc:237] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } }
I20260810 00:04:00.300531 8421 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "13e717dcc6674905ae00b5c37394e840" committed_config { opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } } }
I20260810 00:04:00.300664 8420 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 13e717dcc6674905ae00b5c37394e840. Latest consensus state: current_term: 2 leader_uuid: "13e717dcc6674905ae00b5c37394e840" committed_config { opid_index: -1 peers { permanent_uuid: "13e717dcc6674905ae00b5c37394e840" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 46459 } } }
I20260810 00:04:00.301288 8421 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:00.301324 8420 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:00.303164 8426 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:00.303634 8426 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:04:00.335649 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:46459
I20260810 00:04:00.335821 8459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:46459 every 8 connection(s)
I20260810 00:04:00.341107 8460 heartbeater.cc:347] Connected to a master server at 127.5.100.126:37959
I20260810 00:04:00.341413 5521 tablet_server.cc:178] TabletServer@127.5.100.65:46459 shutting down...
I20260810 00:04:00.341562 8460 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:00.345062 8460 heartbeater.cc:511] Master 127.5.100.126:37959 requested a full tablet report, sending...
I20260810 00:04:00.348184 8191 ts_manager.cc:194] Re-registered known tserver with Master: 13e717dcc6674905ae00b5c37394e840 (127.5.100.65:46459)
W20260810 00:04:00.348708 8191 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:00.350531 8191 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53624
I20260810 00:04:00.353643 8460 heartbeater.cc:503] Master 127.5.100.126:37959 was elected leader, sending a full tablet report...
W20260810 00:04:00.356410 8191 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:00.364405 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:00.365154 5521 tablet_replica.cc:330] stopping tablet replica
I20260810 00:04:00.365957 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 2 LEADER]: Raft consensus shutting down.
I20260810 00:04:00.366413 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 13e717dcc6674905ae00b5c37394e840 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:00.388916 5521 tablet_server.cc:194] TabletServer@127.5.100.65:46459 shutdown complete.
I20260810 00:04:00.401394 5521 master.cc:537] Master@127.5.100.126:37959 shutting down...
I20260810 00:04:00.418543 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:00.419128 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:00.419497 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P d2f0d84650c34ef8a1ec00535a9e816d: stopping tablet replica
I20260810 00:04:00.437577 5521 master.cc:552] Master@127.5.100.126:37959 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (764 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260810 00:04:00.459635 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:46377
I20260810 00:04:00.460641 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:00.465852 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.466024 8467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.469373 8469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:00.469758 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:00.470494 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:00.470686 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:00.470845 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320240470831 us; error 0 us; skew 500 ppm
I20260810 00:04:00.471342 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:00.474200 5521 webserver.cc:460] Webserver started at http://127.5.100.126:42311/ using document root <none> and password file <none>
I20260810 00:04:00.474637 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:00.474823 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:00.475067 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:00.476183 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "08ece2ed086242c6935d4a4f18ad65b5"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.480491 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:04:00.483587 8474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.484373 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:04:00.484704 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "08ece2ed086242c6935d4a4f18ad65b5"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.485029 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:00.511616 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:00.512915 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:00.557607 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:46377
I20260810 00:04:00.557670 8531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:46377 every 8 connection(s)
I20260810 00:04:00.562170 8532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:00.571337 8532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5: Bootstrap starting.
I20260810 00:04:00.575242 8532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:00.579114 8532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5: No bootstrap required, opened a new log
I20260810 00:04:00.580943 8532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER }
I20260810 00:04:00.581353 8532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:00.581588 8532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ece2ed086242c6935d4a4f18ad65b5, State: Initialized, Role: FOLLOWER
I20260810 00:04:00.582152 8532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER }
I20260810 00:04:00.582602 8532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:00.582831 8532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:00.583073 8532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:00.587123 8532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER }
I20260810 00:04:00.587739 8532 leader_election.cc:302] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08ece2ed086242c6935d4a4f18ad65b5; no voters:
I20260810 00:04:00.589071 8532 leader_election.cc:288] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:00.589414 8536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:00.591151 8536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 1 LEADER]: Becoming Leader. State: Replica: 08ece2ed086242c6935d4a4f18ad65b5, State: Running, Role: LEADER
I20260810 00:04:00.591832 8536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER }
I20260810 00:04:00.592180 8532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:04:00.594873 8537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08ece2ed086242c6935d4a4f18ad65b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER } }
I20260810 00:04:00.595532 8537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:00.596944 8538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08ece2ed086242c6935d4a4f18ad65b5. Latest consensus state: current_term: 1 leader_uuid: "08ece2ed086242c6935d4a4f18ad65b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ece2ed086242c6935d4a4f18ad65b5" member_type: VOTER } }
I20260810 00:04:00.597589 8538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:00.607549 8541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:04:00.615149 8541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:04:00.616361 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:04:00.624645 8541 catalog_manager.cc:1347] Generated new cluster ID: 91f84762c95641e3891137847eea5ed7
I20260810 00:04:00.624920 8541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:04:00.645378 8541 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:04:00.647701 8541 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:04:00.664032 8541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5: Generated new TSK 0
I20260810 00:04:00.664692 8541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:04:00.682603 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:00.689174 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.689847 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.692605 8560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:00.693257 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:00.694164 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:00.694368 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:00.694538 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320240694513 us; error 0 us; skew 500 ppm
I20260810 00:04:00.695070 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:00.698004 5521 webserver.cc:460] Webserver started at http://127.5.100.65:34371/ using document root <none> and password file <none>
I20260810 00:04:00.698508 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:00.698685 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:00.698949 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:00.699999 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "a19013485ab044f38603abf637061402"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.704866 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:04:00.708356 8565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.709151 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:04:00.709420 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "a19013485ab044f38603abf637061402"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.709694 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:00.722095 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:00.722996 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:00.735123 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:00.735462 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.735807 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:00.736081 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.811895 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:42907
I20260810 00:04:00.812072 8645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:42907 every 8 connection(s)
I20260810 00:04:00.816627 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 00:04:00.816805 8646 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46377
I20260810 00:04:00.817209 8646 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:00.818023 8646 heartbeater.cc:511] Master 127.5.100.126:46377 requested a full tablet report, sending...
I20260810 00:04:00.820065 8492 ts_manager.cc:194] Registered new tserver with Master: a19013485ab044f38603abf637061402 (127.5.100.65:42907)
I20260810 00:04:00.822734 8492 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44984
W20260810 00:04:00.823442 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.825592 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:00.830186 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:00.830600 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:00.831429 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:00.831599 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:00.831807 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320240831791 us; error 0 us; skew 500 ppm
I20260810 00:04:00.832386 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:00.834928 5521 webserver.cc:460] Webserver started at http://127.5.100.66:33237/ using document root <none> and password file <none>
I20260810 00:04:00.835366 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:00.835534 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:00.835786 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:00.836913 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "2073a615ce4b48b1a06748bf921b3aea"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.841364 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:04:00.844678 8657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.845520 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 00:04:00.845793 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "2073a615ce4b48b1a06748bf921b3aea"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:00.846045 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:00.864655 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:00.865597 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:00.870829 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:00.871096 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.871371 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:00.871567 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:00.936939 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:43335
I20260810 00:04:00.937077 8735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:43335 every 8 connection(s)
I20260810 00:04:00.942026 8736 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46377
I20260810 00:04:00.942453 8736 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:00.943326 8736 heartbeater.cc:511] Master 127.5.100.126:46377 requested a full tablet report, sending...
I20260810 00:04:00.945645 8492 ts_manager.cc:194] Registered new tserver with Master: 2073a615ce4b48b1a06748bf921b3aea (127.5.100.66:43335)
I20260810 00:04:00.946672 5521 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.006300499s
I20260810 00:04:00.947955 8492 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45010
I20260810 00:04:00.988086 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260810 00:04:00.989373 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:01.000193 8747 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Bootstrap starting.
I20260810 00:04:01.004534 8747 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:01.008850 8747 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: No bootstrap required, opened a new log
I20260810 00:04:01.009223 8747 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 00:04:01.011368 8747 raft_consensus.cc:348] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.011770 8747 raft_consensus.cc:374] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:01.011999 8747 raft_consensus.cc:729] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19013485ab044f38603abf637061402, State: Initialized, Role: FOLLOWER
I20260810 00:04:01.012579 8747 consensus_queue.cc:260] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.013042 8747 raft_consensus.cc:388] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:01.013269 8747 raft_consensus.cc:482] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:01.013546 8747 raft_consensus.cc:3037] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:01.018002 8747 raft_consensus.cc:504] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.018695 8747 leader_election.cc:302] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19013485ab044f38603abf637061402; no voters:
I20260810 00:04:01.020041 8747 leader_election.cc:288] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:01.020450 8749 raft_consensus.cc:2781] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:01.022137 8646 heartbeater.cc:503] Master 127.5.100.126:46377 was elected leader, sending a full tablet report...
I20260810 00:04:01.022891 8747 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 00:04:01.023378 8749 raft_consensus.cc:686] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 LEADER]: Becoming Leader. State: Replica: a19013485ab044f38603abf637061402, State: Running, Role: LEADER
I20260810 00:04:01.024086 8749 consensus_queue.cc:237] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.028402 8750 tablet_replica.cc:439] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a19013485ab044f38603abf637061402" committed_config { opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } } }
I20260810 00:04:01.028959 8750 tablet_replica.cc:442] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: This TxnStatusTablet replica's current role is: LEADER
W20260810 00:04:01.030402 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:01.030050 8751 tablet_replica.cc:439] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: TxnStatusTablet state changed. Reason: New leader a19013485ab044f38603abf637061402. Latest consensus state: current_term: 1 leader_uuid: "a19013485ab044f38603abf637061402" committed_config { opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } } }
I20260810 00:04:01.030825 8751 tablet_replica.cc:442] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: This TxnStatusTablet replica's current role is: LEADER
W20260810 00:04:01.036028 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:01.036602 8755 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:01.036952 8755 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:04:01.092684 8759 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Initiating tablet copy from peer a19013485ab044f38603abf637061402 (127.5.100.65:42907)
I20260810 00:04:01.094343 8759 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: tablet copy: Beginning tablet copy session from remote peer at address 127.5.100.65:42907
I20260810 00:04:01.104516 8630 tablet_copy_service.cc:140] P a19013485ab044f38603abf637061402: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer 2073a615ce4b48b1a06748bf921b3aea ({username='slave'} at 127.0.0.1:58564)
I20260810 00:04:01.104956 8630 tablet_copy_service.cc:161] P a19013485ab044f38603abf637061402: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer 2073a615ce4b48b1a06748bf921b3aea at {username='slave'} at 127.0.0.1:58564: session id = 2073a615ce4b48b1a06748bf921b3aea-11111111111111111111111111111111
I20260810 00:04:01.109908 8630 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:04:01.112733 8759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:01.126154 8759 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: tablet copy: Starting download of 0 data blocks...
I20260810 00:04:01.126741 8759 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: tablet copy: Starting download of 1 WAL segments...
I20260810 00:04:01.129814 8759 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:04:01.136605 8759 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Bootstrap starting.
I20260810 00:04:01.163038 8759 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:04:01.163918 8759 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Bootstrap complete.
I20260810 00:04:01.164573 8759 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Time spent bootstrapping tablet: real 0.028s user 0.027s sys 0.000s
I20260810 00:04:01.167022 8759 raft_consensus.cc:348] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.167564 8759 raft_consensus.cc:729] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 2073a615ce4b48b1a06748bf921b3aea, State: Initialized, Role: NON_PARTICIPANT
I20260810 00:04:01.168166 8759 consensus_queue.cc:260] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.169837 8762 tablet_replica.cc:439] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } } }
I20260810 00:04:01.170028 8736 heartbeater.cc:503] Master 127.5.100.126:46377 was elected leader, sending a full tablet report...
I20260810 00:04:01.170478 8762 tablet_replica.cc:442] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260810 00:04:01.170472 8759 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 00:04:01.172374 8630 tablet_copy_service.cc:342] P a19013485ab044f38603abf637061402: Request end of tablet copy session 2073a615ce4b48b1a06748bf921b3aea-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:58564
I20260810 00:04:01.172766 8630 tablet_copy_service.cc:433] P a19013485ab044f38603abf637061402: ending tablet copy session 2073a615ce4b48b1a06748bf921b3aea-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer 2073a615ce4b48b1a06748bf921b3aea
W20260810 00:04:01.172999 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260810 00:04:01.176868 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:01.191026 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:01.223225 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:01.224519 5521 tablet_replica.cc:330] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: stopping tablet replica
I20260810 00:04:01.225159 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:01.225735 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:01.249320 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:01.262727 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:01.270136 8764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.269353 8763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.273484 8766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:01.275353 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:01.276593 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:01.276865 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:01.277043 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320241277027 us; error 0 us; skew 500 ppm
I20260810 00:04:01.277544 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:01.280431 5521 webserver.cc:460] Webserver started at http://127.5.100.65:34371/ using document root <none> and password file <none>
I20260810 00:04:01.280891 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:01.281071 5521 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260810 00:04:01.284672 5521 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 00:04:01.287714 8771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:01.288573 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 00:04:01.288836 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "a19013485ab044f38603abf637061402"
format_stamp: "Formatted at 2026-08-10 00:04:00 on dist-test-slave-8w9v"
I20260810 00:04:01.289129 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:01.313431 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:01.314321 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:01.321779 8782 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 00:04:01.330837 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 00:04:01.331203 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.002s sys 0.000s
I20260810 00:04:01.331506 5521 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 00:04:01.335057 5521 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 00:04:01.335244 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260810 00:04:01.335688 8782 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Bootstrap starting.
I20260810 00:04:01.382591 8782 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:04:01.384460 8782 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Bootstrap complete.
I20260810 00:04:01.385367 8782 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Time spent bootstrapping tablet: real 0.050s user 0.039s sys 0.004s
I20260810 00:04:01.388844 8782 raft_consensus.cc:348] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.389714 8782 raft_consensus.cc:729] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19013485ab044f38603abf637061402, State: Initialized, Role: FOLLOWER
I20260810 00:04:01.390753 8782 consensus_queue.cc:260] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.391681 8782 raft_consensus.cc:388] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:01.392606 8782 raft_consensus.cc:482] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:01.393261 8782 raft_consensus.cc:3037] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 1 FOLLOWER]: Advancing to term 2
I20260810 00:04:01.402029 8782 raft_consensus.cc:504] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.403097 8782 leader_election.cc:302] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19013485ab044f38603abf637061402; no voters:
I20260810 00:04:01.404819 8782 leader_election.cc:288] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 00:04:01.405099 8809 raft_consensus.cc:2781] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:04:01.413594 8809 raft_consensus.cc:686] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 2 LEADER]: Becoming Leader. State: Replica: a19013485ab044f38603abf637061402, State: Running, Role: LEADER
I20260810 00:04:01.414409 8809 consensus_queue.cc:237] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } }
I20260810 00:04:01.414906 8782 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402: Time spent starting tablet: real 0.029s user 0.012s sys 0.014s
I20260810 00:04:01.425503 8811 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a19013485ab044f38603abf637061402" committed_config { opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } } }
I20260810 00:04:01.425843 8812 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader a19013485ab044f38603abf637061402. Latest consensus state: current_term: 2 leader_uuid: "a19013485ab044f38603abf637061402" committed_config { opid_index: -1 peers { permanent_uuid: "a19013485ab044f38603abf637061402" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 42907 } } }
I20260810 00:04:01.426276 8811 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:01.426419 8812 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:01.440510 8820 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:01.441160 8820 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 00:04:01.465826 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:42907
I20260810 00:04:01.466005 8855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:42907 every 8 connection(s)
I20260810 00:04:01.471889 8858 heartbeater.cc:347] Connected to a master server at 127.5.100.126:46377
I20260810 00:04:01.472455 8858 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:01.473342 5521 tablet_server.cc:178] TabletServer@127.5.100.65:42907 shutting down...
I20260810 00:04:01.477957 8858 heartbeater.cc:511] Master 127.5.100.126:46377 requested a full tablet report, sending...
I20260810 00:04:01.482144 8492 ts_manager.cc:194] Re-registered known tserver with Master: a19013485ab044f38603abf637061402 (127.5.100.65:42907)
W20260810 00:04:01.482769 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:01.485457 8492 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45058
I20260810 00:04:01.489024 8858 heartbeater.cc:503] Master 127.5.100.126:46377 was elected leader, sending a full tablet report...
W20260810 00:04:01.492877 8492 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:01.504102 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:01.504878 5521 tablet_replica.cc:330] stopping tablet replica
I20260810 00:04:01.505430 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 2 LEADER]: Raft consensus shutting down.
I20260810 00:04:01.505851 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P a19013485ab044f38603abf637061402 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:01.516508 5521 tablet_server.cc:194] TabletServer@127.5.100.65:42907 shutdown complete.
I20260810 00:04:01.527927 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:04:01.545228 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:01.545904 5521 tablet_replica.cc:330] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea: stopping tablet replica
I20260810 00:04:01.546536 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 00:04:01.546954 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 2073a615ce4b48b1a06748bf921b3aea [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 00:04:01.566541 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:04:01.576907 5521 master.cc:537] Master@127.5.100.126:46377 shutting down...
I20260810 00:04:01.593930 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:01.594420 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:01.594704 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 08ece2ed086242c6935d4a4f18ad65b5: stopping tablet replica
I20260810 00:04:01.612844 5521 master.cc:552] Master@127.5.100.126:46377 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1177 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260810 00:04:01.637516 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:33849
I20260810 00:04:01.638610 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:01.643951 8864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.644635 8865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.647518 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:01.648424 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:01.649119 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:01.649310 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:01.649466 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320241649452 us; error 0 us; skew 500 ppm
I20260810 00:04:01.649978 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:01.652869 5521 webserver.cc:460] Webserver started at http://127.5.100.126:37633/ using document root <none> and password file <none>
I20260810 00:04:01.653312 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:01.653501 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:01.653748 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:01.654806 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "dd088396072b4582ba91f25ffeac18c9"
format_stamp: "Formatted at 2026-08-10 00:04:01 on dist-test-slave-8w9v"
I20260810 00:04:01.659341 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:04:01.662701 8873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:01.663528 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:04:01.663779 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "dd088396072b4582ba91f25ffeac18c9"
format_stamp: "Formatted at 2026-08-10 00:04:01 on dist-test-slave-8w9v"
I20260810 00:04:01.664027 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:01.682936 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:01.683929 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:01.733909 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:33849
I20260810 00:04:01.734123 8927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:33849 every 8 connection(s)
I20260810 00:04:01.738274 8928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:01.753042 8928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9: Bootstrap starting.
I20260810 00:04:01.758844 8928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:01.763491 8928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9: No bootstrap required, opened a new log
I20260810 00:04:01.766103 8928 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER }
I20260810 00:04:01.766640 8928 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:01.766916 8928 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd088396072b4582ba91f25ffeac18c9, State: Initialized, Role: FOLLOWER
I20260810 00:04:01.767549 8928 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER }
I20260810 00:04:01.768162 8928 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:01.768445 8928 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:01.768741 8928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:01.775172 8928 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER }
I20260810 00:04:01.775789 8928 leader_election.cc:302] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd088396072b4582ba91f25ffeac18c9; no voters:
I20260810 00:04:01.777201 8928 leader_election.cc:288] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:01.777488 8933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:01.779418 8933 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 1 LEADER]: Becoming Leader. State: Replica: dd088396072b4582ba91f25ffeac18c9, State: Running, Role: LEADER
I20260810 00:04:01.780068 8933 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER }
I20260810 00:04:01.780517 8928 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:04:01.782879 8934 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd088396072b4582ba91f25ffeac18c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER } }
I20260810 00:04:01.783303 8934 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:01.784022 8935 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd088396072b4582ba91f25ffeac18c9. Latest consensus state: current_term: 1 leader_uuid: "dd088396072b4582ba91f25ffeac18c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd088396072b4582ba91f25ffeac18c9" member_type: VOTER } }
I20260810 00:04:01.784652 8935 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:01.797911 8939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:04:01.803436 8939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:04:01.807220 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:04:01.813495 8939 catalog_manager.cc:1347] Generated new cluster ID: 8a30b9f02f6e46b38709f29159426713
I20260810 00:04:01.813752 8939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:04:01.828657 8939 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:04:01.830258 8939 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:04:01.844316 8939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9: Generated new TSK 0
I20260810 00:04:01.844993 8939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:04:01.876058 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:01.882309 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.884650 8954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:01.885097 8956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:01.885876 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:01.886857 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:01.887120 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:01.887300 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320241887287 us; error 0 us; skew 500 ppm
I20260810 00:04:01.888016 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:01.890686 5521 webserver.cc:460] Webserver started at http://127.5.100.65:38787/ using document root <none> and password file <none>
I20260810 00:04:01.891268 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:01.891484 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:01.891782 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:01.893204 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "cf8a9dc135f14dacbb810130bbd379ca"
format_stamp: "Formatted at 2026-08-10 00:04:01 on dist-test-slave-8w9v"
I20260810 00:04:01.899189 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 00:04:01.903520 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:01.904381 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:04:01.904707 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "cf8a9dc135f14dacbb810130bbd379ca"
format_stamp: "Formatted at 2026-08-10 00:04:01 on dist-test-slave-8w9v"
I20260810 00:04:01.905057 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:01.957015 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:01.959292 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:01.969340 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:01.969693 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:01.970098 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:01.970330 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:02.050233 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:38791
I20260810 00:04:02.050390 9043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:38791 every 8 connection(s)
I20260810 00:04:02.057921 9044 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33849
I20260810 00:04:02.058660 9044 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:02.060060 9044 heartbeater.cc:511] Master 127.5.100.126:33849 requested a full tablet report, sending...
I20260810 00:04:02.063289 8891 ts_manager.cc:194] Registered new tserver with Master: cf8a9dc135f14dacbb810130bbd379ca (127.5.100.65:38791)
I20260810 00:04:02.063571 5521 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.007280248s
I20260810 00:04:02.067601 8891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50706
I20260810 00:04:02.112531 9005 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260810 00:04:02.114131 9005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:02.128902 9055 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: Bootstrap starting.
I20260810 00:04:02.134227 9055 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:02.138693 9055 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: No bootstrap required, opened a new log
I20260810 00:04:02.139043 9055 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260810 00:04:02.141094 9055 raft_consensus.cc:348] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } }
I20260810 00:04:02.141870 9055 raft_consensus.cc:374] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:02.142083 9055 raft_consensus.cc:729] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8a9dc135f14dacbb810130bbd379ca, State: Initialized, Role: FOLLOWER
I20260810 00:04:02.142607 9055 consensus_queue.cc:260] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } }
I20260810 00:04:02.143080 9055 raft_consensus.cc:388] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:02.143317 9055 raft_consensus.cc:482] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:02.143599 9055 raft_consensus.cc:3037] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:02.148734 9055 raft_consensus.cc:504] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } }
I20260810 00:04:02.149235 9055 leader_election.cc:302] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf8a9dc135f14dacbb810130bbd379ca; no voters:
I20260810 00:04:02.150483 9055 leader_election.cc:288] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:02.151121 9057 raft_consensus.cc:2781] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:02.152590 9044 heartbeater.cc:503] Master 127.5.100.126:33849 was elected leader, sending a full tablet report...
I20260810 00:04:02.153136 9055 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260810 00:04:02.153774 9057 raft_consensus.cc:686] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 1 LEADER]: Becoming Leader. State: Replica: cf8a9dc135f14dacbb810130bbd379ca, State: Running, Role: LEADER
I20260810 00:04:02.154587 9057 consensus_queue.cc:237] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } }
I20260810 00:04:02.161286 9058 tablet_replica.cc:439] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf8a9dc135f14dacbb810130bbd379ca" committed_config { opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } } }
I20260810 00:04:02.161407 9059 tablet_replica.cc:439] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: TxnStatusTablet state changed. Reason: New leader cf8a9dc135f14dacbb810130bbd379ca. Latest consensus state: current_term: 1 leader_uuid: "cf8a9dc135f14dacbb810130bbd379ca" committed_config { opid_index: -1 peers { permanent_uuid: "cf8a9dc135f14dacbb810130bbd379ca" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38791 } } }
I20260810 00:04:02.161912 9059 tablet_replica.cc:442] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:02.161888 9058 tablet_replica.cc:442] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: This TxnStatusTablet replica's current role is: LEADER
W20260810 00:04:02.167443 8891 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:02.168020 9062 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:02.168583 9062 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260810 00:04:02.173163 8891 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260810 00:04:02.229789 8978 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260810 00:04:02.230180 8978 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260810 00:04:02.230589 8978 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260810 00:04:02.239625 8978 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260810 00:04:02.281139 8978 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260810 00:04:02.281418 8978 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260810 00:04:02.290858 8978 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260810 00:04:02.305583 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.307852 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.309965 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.312023 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.314256 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.316404 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:50956
W20260810 00:04:02.328755 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
W20260810 00:04:02.330682 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
W20260810 00:04:02.332548 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
W20260810 00:04:02.334488 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
W20260810 00:04:02.336329 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
W20260810 00:04:02.338145 9004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:50964
I20260810 00:04:02.341328 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:02.370996 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:02.371814 5521 tablet_replica.cc:330] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca: stopping tablet replica
I20260810 00:04:02.372375 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:02.372859 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P cf8a9dc135f14dacbb810130bbd379ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:02.385696 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:02.400889 5521 master.cc:537] Master@127.5.100.126:33849 shutting down...
I20260810 00:04:02.432480 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:02.433346 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:02.433760 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P dd088396072b4582ba91f25ffeac18c9: stopping tablet replica
I20260810 00:04:02.453954 5521 master.cc:552] Master@127.5.100.126:33849 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (856 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260810 00:04:02.496284 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:34775
I20260810 00:04:02.497756 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:02.504695 9070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:02.504854 9072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:02.507735 9074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:02.513490 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:02.514418 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:02.514614 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:02.514859 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320242514815 us; error 0 us; skew 500 ppm
I20260810 00:04:02.515441 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:02.518543 5521 webserver.cc:460] Webserver started at http://127.5.100.126:37579/ using document root <none> and password file <none>
I20260810 00:04:02.519048 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:02.519217 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:02.519515 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:02.520793 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "d018529ea4674f8facf9673060d69b4b"
format_stamp: "Formatted at 2026-08-10 00:04:02 on dist-test-slave-8w9v"
I20260810 00:04:02.525660 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:04:02.529515 9079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:02.530406 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 00:04:02.530741 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "d018529ea4674f8facf9673060d69b4b"
format_stamp: "Formatted at 2026-08-10 00:04:02 on dist-test-slave-8w9v"
I20260810 00:04:02.531152 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:02.543354 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:02.544428 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:02.580353 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:34775
I20260810 00:04:02.580490 9134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:34775 every 8 connection(s)
I20260810 00:04:02.583787 9135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:02.593997 9135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b: Bootstrap starting.
I20260810 00:04:02.598289 9135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:02.601917 9135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b: No bootstrap required, opened a new log
I20260810 00:04:02.603636 9135 raft_consensus.cc:348] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER }
I20260810 00:04:02.603972 9135 raft_consensus.cc:374] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:02.604166 9135 raft_consensus.cc:729] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d018529ea4674f8facf9673060d69b4b, State: Initialized, Role: FOLLOWER
I20260810 00:04:02.604703 9135 consensus_queue.cc:260] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER }
I20260810 00:04:02.605098 9135 raft_consensus.cc:388] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:02.605259 9135 raft_consensus.cc:482] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:02.605438 9135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:02.609223 9135 raft_consensus.cc:504] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER }
I20260810 00:04:02.609638 9135 leader_election.cc:302] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d018529ea4674f8facf9673060d69b4b; no voters:
I20260810 00:04:02.610638 9135 leader_election.cc:288] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:02.610975 9140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:02.612203 9140 raft_consensus.cc:686] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 1 LEADER]: Becoming Leader. State: Replica: d018529ea4674f8facf9673060d69b4b, State: Running, Role: LEADER
I20260810 00:04:02.612787 9140 consensus_queue.cc:237] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER }
I20260810 00:04:02.613341 9135 sys_catalog.cc:564] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:04:02.615464 9141 sys_catalog.cc:455] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d018529ea4674f8facf9673060d69b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER } }
I20260810 00:04:02.615568 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d018529ea4674f8facf9673060d69b4b. Latest consensus state: current_term: 1 leader_uuid: "d018529ea4674f8facf9673060d69b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d018529ea4674f8facf9673060d69b4b" member_type: VOTER } }
I20260810 00:04:02.616339 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:02.616912 9141 sys_catalog.cc:458] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:02.619091 9147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:04:02.624845 9147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:04:02.629707 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:04:02.633309 9147 catalog_manager.cc:1347] Generated new cluster ID: 5e549080e9cb4fa2bf2ae366d573845b
I20260810 00:04:02.633632 9147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:04:02.659294 9147 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:04:02.660909 9147 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:04:02.675933 9147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b: Generated new TSK 0
I20260810 00:04:02.676576 9147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:04:02.696182 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:02.701606 9161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:02.702675 9162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:02.705098 5521 server_base.cc:1034] running on GCE node
W20260810 00:04:02.705541 9164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:02.706422 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:02.706597 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:02.706699 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320242706692 us; error 0 us; skew 500 ppm
I20260810 00:04:02.707110 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:02.709266 5521 webserver.cc:460] Webserver started at http://127.5.100.65:44475/ using document root <none> and password file <none>
I20260810 00:04:02.709678 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:02.709829 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:02.710018 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:02.711043 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "258dcbf0a2c44f0690bebfadceae20fc"
format_stamp: "Formatted at 2026-08-10 00:04:02 on dist-test-slave-8w9v"
I20260810 00:04:02.715545 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 00:04:02.718680 9170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:02.719373 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 00:04:02.719659 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "258dcbf0a2c44f0690bebfadceae20fc"
format_stamp: "Formatted at 2026-08-10 00:04:02 on dist-test-slave-8w9v"
I20260810 00:04:02.719926 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:02.729025 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:02.729987 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:02.735351 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:02.735692 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:02.736043 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:02.736331 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:02.788564 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:38481
I20260810 00:04:02.788712 9242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:38481 every 8 connection(s)
I20260810 00:04:02.793234 9243 heartbeater.cc:347] Connected to a master server at 127.5.100.126:34775
I20260810 00:04:02.793602 9243 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:02.794332 9243 heartbeater.cc:511] Master 127.5.100.126:34775 requested a full tablet report, sending...
I20260810 00:04:02.796204 9098 ts_manager.cc:194] Registered new tserver with Master: 258dcbf0a2c44f0690bebfadceae20fc (127.5.100.65:38481)
I20260810 00:04:02.796649 5521 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00484251s
I20260810 00:04:02.798724 9098 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48334
I20260810 00:04:02.830593 9208 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260810 00:04:02.831722 9208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:02.842711 9257 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: Bootstrap starting.
I20260810 00:04:02.846917 9257 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:02.851022 9257 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: No bootstrap required, opened a new log
I20260810 00:04:02.851373 9257 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:04:02.853204 9257 raft_consensus.cc:348] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.853624 9257 raft_consensus.cc:374] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:02.853839 9257 raft_consensus.cc:729] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258dcbf0a2c44f0690bebfadceae20fc, State: Initialized, Role: FOLLOWER
I20260810 00:04:02.854321 9257 consensus_queue.cc:260] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.854773 9257 raft_consensus.cc:388] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:02.855003 9257 raft_consensus.cc:482] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:02.855247 9257 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:02.859447 9257 raft_consensus.cc:504] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.859949 9257 leader_election.cc:302] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258dcbf0a2c44f0690bebfadceae20fc; no voters:
I20260810 00:04:02.861048 9257 leader_election.cc:288] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:02.861369 9260 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:02.863155 9257 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 00:04:02.862980 9243 heartbeater.cc:503] Master 127.5.100.126:34775 was elected leader, sending a full tablet report...
I20260810 00:04:02.863430 9260 raft_consensus.cc:686] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 LEADER]: Becoming Leader. State: Replica: 258dcbf0a2c44f0690bebfadceae20fc, State: Running, Role: LEADER
I20260810 00:04:02.864182 9260 consensus_queue.cc:237] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.868178 9261 tablet_replica.cc:439] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "258dcbf0a2c44f0690bebfadceae20fc" committed_config { opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } } }
I20260810 00:04:02.868818 9261 tablet_replica.cc:442] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:02.870426 9265 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 00:04:02.870255 9262 tablet_replica.cc:439] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: TxnStatusTablet state changed. Reason: New leader 258dcbf0a2c44f0690bebfadceae20fc. Latest consensus state: current_term: 1 leader_uuid: "258dcbf0a2c44f0690bebfadceae20fc" committed_config { opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } } }
I20260810 00:04:02.870810 9262 tablet_replica.cc:442] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: This TxnStatusTablet replica's current role is: LEADER
I20260810 00:04:02.870910 9265 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260810 00:04:02.871830 9098 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260810 00:04:02.875941 9098 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260810 00:04:02.893863 9208 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260810 00:04:02.894909 9208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:02.904661 9257 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: Bootstrap starting.
I20260810 00:04:02.908938 9257 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:02.912859 9257 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: No bootstrap required, opened a new log
I20260810 00:04:02.913229 9257 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 00:04:02.915095 9257 raft_consensus.cc:348] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.915513 9257 raft_consensus.cc:374] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:02.915731 9257 raft_consensus.cc:729] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258dcbf0a2c44f0690bebfadceae20fc, State: Initialized, Role: FOLLOWER
I20260810 00:04:02.916302 9257 consensus_queue.cc:260] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.916764 9257 raft_consensus.cc:388] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:02.916992 9257 raft_consensus.cc:482] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:02.917251 9257 raft_consensus.cc:3037] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:02.921720 9257 raft_consensus.cc:504] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
I20260810 00:04:02.922293 9257 leader_election.cc:302] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258dcbf0a2c44f0690bebfadceae20fc; no voters:
I20260810 00:04:02.922832 9257 leader_election.cc:288] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:02.923022 9262 raft_consensus.cc:2781] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:02.923524 9262 raft_consensus.cc:686] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 LEADER]: Becoming Leader. State: Replica: 258dcbf0a2c44f0690bebfadceae20fc, State: Running, Role: LEADER
I20260810 00:04:02.924283 9257 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 00:04:02.924216 9262 consensus_queue.cc:237] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "258dcbf0a2c44f0690bebfadceae20fc" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 38481 } }
W20260810 00:04:02.929157 9098 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260810 00:04:02.932826 9098 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260810 00:04:02.953509 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:02.974026 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:02.974776 5521 tablet_replica.cc:330] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc: stopping tablet replica
I20260810 00:04:02.975291 5521 raft_consensus.cc:2229] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:02.975732 5521 raft_consensus.cc:2258] T 22222222222222222222222222222222 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:02.978344 5521 tablet_replica.cc:330] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc: stopping tablet replica
I20260810 00:04:02.978816 5521 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:02.979236 5521 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 258dcbf0a2c44f0690bebfadceae20fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:02.991029 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:03.004274 5521 master.cc:537] Master@127.5.100.126:34775 shutting down...
I20260810 00:04:03.024173 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:03.025084 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:03.025698 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P d018529ea4674f8facf9673060d69b4b: stopping tablet replica
I20260810 00:04:03.036744 5521 master.cc:552] Master@127.5.100.126:34775 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (571 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3369 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260810 00:04:03.065459 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:39657
I20260810 00:04:03.066646 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:03.071529 9276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.072782 9277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.074265 9281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:03.074419 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:03.075292 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:03.075456 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:03.075562 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320243075554 us; error 0 us; skew 500 ppm
I20260810 00:04:03.076026 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:03.078320 5521 webserver.cc:460] Webserver started at http://127.5.100.126:42443/ using document root <none> and password file <none>
I20260810 00:04:03.078748 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:03.078902 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:03.079115 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:03.080533 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "ce3398b457924d618a4c44fed9b4ffc8"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.085040 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 00:04:03.088745 9286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.089494 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 00:04:03.089733 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "ce3398b457924d618a4c44fed9b4ffc8"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.089983 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:03.117434 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:03.118350 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:03.152472 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:39657
I20260810 00:04:03.152568 9340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:39657 every 8 connection(s)
I20260810 00:04:03.156061 9341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:03.166383 9341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8: Bootstrap starting.
I20260810 00:04:03.170414 9341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:03.174046 9341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8: No bootstrap required, opened a new log
I20260810 00:04:03.175832 9341 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER }
I20260810 00:04:03.176282 9341 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:03.176497 9341 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3398b457924d618a4c44fed9b4ffc8, State: Initialized, Role: FOLLOWER
I20260810 00:04:03.177083 9341 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER }
I20260810 00:04:03.177525 9341 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:03.177732 9341 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:03.178014 9341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:03.181905 9341 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER }
I20260810 00:04:03.182386 9341 leader_election.cc:302] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce3398b457924d618a4c44fed9b4ffc8; no voters:
I20260810 00:04:03.183429 9341 leader_election.cc:288] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:03.183727 9345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:03.185062 9345 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 1 LEADER]: Becoming Leader. State: Replica: ce3398b457924d618a4c44fed9b4ffc8, State: Running, Role: LEADER
I20260810 00:04:03.185629 9345 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER }
I20260810 00:04:03.186262 9341 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:04:03.188660 9347 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce3398b457924d618a4c44fed9b4ffc8. Latest consensus state: current_term: 1 leader_uuid: "ce3398b457924d618a4c44fed9b4ffc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER } }
I20260810 00:04:03.188628 9346 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce3398b457924d618a4c44fed9b4ffc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3398b457924d618a4c44fed9b4ffc8" member_type: VOTER } }
I20260810 00:04:03.190124 9347 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:03.190222 9346 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:03.191982 9353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:04:03.198118 9353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:04:03.199671 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:04:03.208988 9353 catalog_manager.cc:1347] Generated new cluster ID: 07ad09fce16e402a8472e4906bfb93bc
I20260810 00:04:03.209273 9353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:04:03.230477 9353 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:04:03.232060 9353 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:04:03.244035 9353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8: Generated new TSK 0
I20260810 00:04:03.244724 9353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:04:03.265843 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:03.271476 9367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.273054 9368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.275200 9370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:03.275269 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:03.276319 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:03.276520 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:03.276671 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320243276653 us; error 0 us; skew 500 ppm
I20260810 00:04:03.277160 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:03.279455 5521 webserver.cc:460] Webserver started at http://127.5.100.65:34277/ using document root <none> and password file <none>
I20260810 00:04:03.279927 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:03.280174 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:03.280454 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:03.281490 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "1c98c733670540aeaac8ec6e489bd61d"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.285876 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:04:03.289206 9375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.290015 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:04:03.290387 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "1c98c733670540aeaac8ec6e489bd61d"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.290686 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:03.317302 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:03.318338 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:03.319792 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:03.322089 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:03.322304 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.322546 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:03.322717 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.360863 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:34035
I20260810 00:04:03.360987 9447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:34035 every 8 connection(s)
I20260810 00:04:03.366153 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:03.373930 9452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.374476 9453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.377198 9456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:03.382515 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:03.382809 9448 heartbeater.cc:347] Connected to a master server at 127.5.100.126:39657
I20260810 00:04:03.383174 9448 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:03.383531 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:03.383764 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:03.383836 9448 heartbeater.cc:511] Master 127.5.100.126:39657 requested a full tablet report, sending...
I20260810 00:04:03.383944 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320243383928 us; error 0 us; skew 500 ppm
I20260810 00:04:03.384583 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:03.385756 9305 ts_manager.cc:194] Registered new tserver with Master: 1c98c733670540aeaac8ec6e489bd61d (127.5.100.65:34035)
I20260810 00:04:03.387392 5521 webserver.cc:460] Webserver started at http://127.5.100.66:40819/ using document root <none> and password file <none>
I20260810 00:04:03.387935 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:03.387971 9305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49678
I20260810 00:04:03.388217 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:03.388576 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:03.389809 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "bb54794c45954644b6c8ac8f6d388d5c"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.394563 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:04:03.397852 9461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.398612 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:04:03.398912 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "bb54794c45954644b6c8ac8f6d388d5c"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.399187 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:03.409292 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:03.410190 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:03.411453 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:03.413571 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:03.413781 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.414011 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:03.414163 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.452451 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:38173
I20260810 00:04:03.452551 9526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:38173 every 8 connection(s)
I20260810 00:04:03.456822 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:03.464854 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.466459 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:03.467415 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:03.467697 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:03.468770 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:03.469020 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:03.469244 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320243469228 us; error 0 us; skew 500 ppm
I20260810 00:04:03.469901 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:03.471781 9527 heartbeater.cc:347] Connected to a master server at 127.5.100.126:39657
I20260810 00:04:03.472160 9527 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:03.472800 5521 webserver.cc:460] Webserver started at http://127.5.100.67:44727/ using document root <none> and password file <none>
I20260810 00:04:03.472918 9527 heartbeater.cc:511] Master 127.5.100.126:39657 requested a full tablet report, sending...
I20260810 00:04:03.473338 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:03.473560 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:03.473856 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:03.475051 9305 ts_manager.cc:194] Registered new tserver with Master: bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173)
I20260810 00:04:03.475368 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root/instance:
uuid: "a3f1f167658f46c49b5a002d88d7348c"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.477254 9305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33000
I20260810 00:04:03.480552 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 00:04:03.483513 9538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.484215 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:04:03.484477 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root
uuid: "a3f1f167658f46c49b5a002d88d7348c"
format_stamp: "Formatted at 2026-08-10 00:04:03 on dist-test-slave-8w9v"
I20260810 00:04:03.484760 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:03.506313 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:03.507382 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:03.508822 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:03.510943 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:03.511116 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.511380 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:03.511528 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:03.560175 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.67:43827
I20260810 00:04:03.560302 9611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.67:43827 every 8 connection(s)
I20260810 00:04:03.578830 9612 heartbeater.cc:347] Connected to a master server at 127.5.100.126:39657
I20260810 00:04:03.579208 9612 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:03.579878 9612 heartbeater.cc:511] Master 127.5.100.126:39657 requested a full tablet report, sending...
I20260810 00:04:03.582036 9305 ts_manager.cc:194] Registered new tserver with Master: a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827)
I20260810 00:04:03.582976 5521 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018384937s
I20260810 00:04:03.584141 9305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33014
I20260810 00:04:03.605476 5521 test_util.cc:268] Using random seed: 395526093
I20260810 00:04:03.627961 9305 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33026:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:04:03.675540 9574 tablet_service.cc:1467] Processing CreateTablet for tablet 518b732f104b4a80b1d63d0222eaf87b (DEFAULT_TABLE table=test-workload [id=5d97f0fe62ed450eafdfa0312859ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:03.676940 9574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518b732f104b4a80b1d63d0222eaf87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:03.693370 9492 tablet_service.cc:1467] Processing CreateTablet for tablet 518b732f104b4a80b1d63d0222eaf87b (DEFAULT_TABLE table=test-workload [id=5d97f0fe62ed450eafdfa0312859ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:03.694581 9492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518b732f104b4a80b1d63d0222eaf87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:03.695479 9633 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Bootstrap starting.
I20260810 00:04:03.697293 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 518b732f104b4a80b1d63d0222eaf87b (DEFAULT_TABLE table=test-workload [id=5d97f0fe62ed450eafdfa0312859ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:03.698563 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518b732f104b4a80b1d63d0222eaf87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:03.701207 9633 tablet_bootstrap.cc:654] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:03.706303 9633 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: No bootstrap required, opened a new log
I20260810 00:04:03.706676 9633 ts_tablet_manager.cc:1397] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260810 00:04:03.708815 9633 raft_consensus.cc:348] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.709389 9633 raft_consensus.cc:374] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:03.709673 9633 raft_consensus.cc:729] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f1f167658f46c49b5a002d88d7348c, State: Initialized, Role: FOLLOWER
I20260810 00:04:03.710471 9633 consensus_queue.cc:260] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.711499 9635 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: Bootstrap starting.
I20260810 00:04:03.713817 9637 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Bootstrap starting.
I20260810 00:04:03.715369 9612 heartbeater.cc:503] Master 127.5.100.126:39657 was elected leader, sending a full tablet report...
I20260810 00:04:03.716635 9633 ts_tablet_manager.cc:1428] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260810 00:04:03.718917 9635 tablet_bootstrap.cc:654] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:03.720439 9637 tablet_bootstrap.cc:654] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:03.726038 9637 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: No bootstrap required, opened a new log
I20260810 00:04:03.726089 9635 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: No bootstrap required, opened a new log
I20260810 00:04:03.726536 9637 ts_tablet_manager.cc:1397] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260810 00:04:03.726624 9635 ts_tablet_manager.cc:1397] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260810 00:04:03.728881 9637 raft_consensus.cc:348] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.729590 9637 raft_consensus.cc:374] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:03.729342 9635 raft_consensus.cc:348] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.729813 9637 raft_consensus.cc:729] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c98c733670540aeaac8ec6e489bd61d, State: Initialized, Role: FOLLOWER
I20260810 00:04:03.729955 9635 raft_consensus.cc:374] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:03.730269 9635 raft_consensus.cc:729] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb54794c45954644b6c8ac8f6d388d5c, State: Initialized, Role: FOLLOWER
I20260810 00:04:03.730523 9637 consensus_queue.cc:260] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.730759 9635 consensus_queue.cc:260] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.733075 9448 heartbeater.cc:503] Master 127.5.100.126:39657 was elected leader, sending a full tablet report...
I20260810 00:04:03.733108 9527 heartbeater.cc:503] Master 127.5.100.126:39657 was elected leader, sending a full tablet report...
I20260810 00:04:03.733467 9637 ts_tablet_manager.cc:1428] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260810 00:04:03.735411 9635 ts_tablet_manager.cc:1428] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260810 00:04:03.791872 9640 raft_consensus.cc:482] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:04:03.792474 9640 raft_consensus.cc:504] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.794555 9640 leader_election.cc:288] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827), bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173)
I20260810 00:04:03.806437 9585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518b732f104b4a80b1d63d0222eaf87b" candidate_uuid: "1c98c733670540aeaac8ec6e489bd61d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f1f167658f46c49b5a002d88d7348c" is_pre_election: true
I20260810 00:04:03.807377 9585 raft_consensus.cc:2454] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c98c733670540aeaac8ec6e489bd61d in term 0.
I20260810 00:04:03.808075 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518b732f104b4a80b1d63d0222eaf87b" candidate_uuid: "1c98c733670540aeaac8ec6e489bd61d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb54794c45954644b6c8ac8f6d388d5c" is_pre_election: true
I20260810 00:04:03.808781 9502 raft_consensus.cc:2454] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c98c733670540aeaac8ec6e489bd61d in term 0.
I20260810 00:04:03.808724 9379 leader_election.cc:302] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c98c733670540aeaac8ec6e489bd61d, a3f1f167658f46c49b5a002d88d7348c; no voters:
I20260810 00:04:03.809597 9640 raft_consensus.cc:2781] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:04:03.809827 9640 raft_consensus.cc:482] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:04:03.810020 9640 raft_consensus.cc:3037] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:03.814845 9640 raft_consensus.cc:504] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.816439 9640 leader_election.cc:288] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [CANDIDATE]: Term 1 election: Requested vote from peers a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827), bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173)
I20260810 00:04:03.817058 9585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518b732f104b4a80b1d63d0222eaf87b" candidate_uuid: "1c98c733670540aeaac8ec6e489bd61d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f1f167658f46c49b5a002d88d7348c"
I20260810 00:04:03.817400 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518b732f104b4a80b1d63d0222eaf87b" candidate_uuid: "1c98c733670540aeaac8ec6e489bd61d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb54794c45954644b6c8ac8f6d388d5c"
I20260810 00:04:03.817654 9585 raft_consensus.cc:3037] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:03.817943 9502 raft_consensus.cc:3037] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:03.824108 9502 raft_consensus.cc:2454] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c98c733670540aeaac8ec6e489bd61d in term 1.
I20260810 00:04:03.824108 9585 raft_consensus.cc:2454] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c98c733670540aeaac8ec6e489bd61d in term 1.
I20260810 00:04:03.825337 9379 leader_election.cc:302] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c98c733670540aeaac8ec6e489bd61d, a3f1f167658f46c49b5a002d88d7348c; no voters:
I20260810 00:04:03.826002 9640 raft_consensus.cc:2781] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:03.826876 9640 raft_consensus.cc:686] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 LEADER]: Becoming Leader. State: Replica: 1c98c733670540aeaac8ec6e489bd61d, State: Running, Role: LEADER
I20260810 00:04:03.827531 9640 consensus_queue.cc:237] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:03.833657 9305 catalog_manager.cc:5495] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c98c733670540aeaac8ec6e489bd61d (127.5.100.65). New cstate: current_term: 1 leader_uuid: "1c98c733670540aeaac8ec6e489bd61d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } }
I20260810 00:04:04.073365 9405 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:04.079245 9492 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:04.080055 9574 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:04.239392 9640 consensus_queue.cc:1035] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:04:04.255801 9640 consensus_queue.cc:1035] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
ce3398b457924d618a4c44fed9b4ffc8 | 127.5.100.126:39657 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c98c733670540aeaac8ec6e489bd61d | 127.5.100.65:34035 | HEALTHY | <none> | 1 | 0
a3f1f167658f46c49b5a002d88d7348c | 127.5.100.67:43827 | HEALTHY | <none> | 0 | 0
bb54794c45954644b6c8ac8f6d388d5c | 127.5.100.66:38173 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 518b732f104b4a80b1d63d0222eaf87b of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): RUNNING
1c98c733670540aeaac8ec6e489bd61d (127.5.100.65:34035): RUNNING [LEADER]
bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173): RUNNING
All reported replicas are:
A = a3f1f167658f46c49b5a002d88d7348c
B = 1c98c733670540aeaac8ec6e489bd61d
C = bb54794c45954644b6c8ac8f6d388d5c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:04.310827 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:04.673233 9405 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:04.683257 9574 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:04.685128 9492 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
ce3398b457924d618a4c44fed9b4ffc8 | 127.5.100.126:39657 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c98c733670540aeaac8ec6e489bd61d | 127.5.100.65:34035 | HEALTHY | <none> | 1 | 0
a3f1f167658f46c49b5a002d88d7348c | 127.5.100.67:43827 | HEALTHY | <none> | 0 | 0
bb54794c45954644b6c8ac8f6d388d5c | 127.5.100.66:38173 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260810 00:04:04.867129 5521 log_verifier.cc:126] Checking tablet 518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:04.926934 5521 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:04.927541 5521 tablet_replica.cc:330] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: stopping tablet replica
I20260810 00:04:04.928094 5521 raft_consensus.cc:2229] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:04:04.928604 5521 raft_consensus.cc:2258] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:05.131495 9405 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:05.136078 9492 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:05.140352 9574 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20260810 00:04:05.308706 9379 consensus_peers.cc:487] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d -> Peer a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): Couldn't send request to peer a3f1f167658f46c49b5a002d88d7348c. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:04:05.310001 9640 raft_consensus.cc:998] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Attempting to remove follower a3f1f167658f46c49b5a002d88d7348c from the Raft config. Reason: The tablet replica hosted on peer a3f1f167658f46c49b5a002d88d7348c has failed
I20260810 00:04:05.311882 9640 consensus_queue.cc:237] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } }
I20260810 00:04:05.316406 9502 raft_consensus.cc:1261] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Refusing update from remote peer 1c98c733670540aeaac8ec6e489bd61d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 00:04:05.317596 9737 consensus_queue.cc:1035] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
ce3398b457924d618a4c44fed9b4ffc8 | 127.5.100.126:39657 | HEALTHY
Unusual flags for Master:
I20260810 00:04:05.324178 9640 raft_consensus.cc:2932] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } }
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c98c733670540aeaac8ec6e489bd61d | 127.5.100.65:34035 | HEALTHY | <none> | 1 | 0
a3f1f167658f46c49b5a002d88d7348c | 127.5.100.67:43827 | HEALTHY | <none> | 0 | 0
bb54794c45954644b6c8ac8f6d388d5c | 127.5.100.66:38173 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
I20260810 00:04:05.325655 9502 raft_consensus.cc:2932] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } }
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 518b732f104b4a80b1d63d0222eaf87b of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
1c98c733670540aeaac8ec6e489bd61d (127.5.100.65:34035): RUNNING [LEADER]
bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173): RUNNING
All reported replicas are:
A = a3f1f167658f46c49b5a002d88d7348c
B = 1c98c733670540aeaac8ec6e489bd61d
C = bb54794c45954644b6c8ac8f6d388d5c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | [config not available] | | |
B | A B* C | 1 | -1 | Yes
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:05.336755 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:05.339802 9302 catalog_manager.cc:5495] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d reported cstate change: config changed from index -1 to 2, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) evicted. New cstate: current_term: 1 leader_uuid: "1c98c733670540aeaac8ec6e489bd61d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } }
I20260810 00:04:05.349828 9574 tablet_service.cc:1514] Processing DeleteTablet for tablet 518b732f104b4a80b1d63d0222eaf87b with delete_type TABLET_DATA_TOMBSTONED (TS a3f1f167658f46c49b5a002d88d7348c not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:55982
I20260810 00:04:05.351274 9747 ts_tablet_manager.cc:1905] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260810 00:04:05.352500 9418 consensus_queue.cc:237] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } }
I20260810 00:04:05.356621 9502 raft_consensus.cc:1261] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Refusing update from remote peer 1c98c733670540aeaac8ec6e489bd61d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260810 00:04:05.357190 9379 consensus_peers.cc:487] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d -> Peer a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): Couldn't send request to peer a3f1f167658f46c49b5a002d88d7348c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:04:05.357805 9737 consensus_queue.cc:1035] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 00:04:05.363015 9747 ts_tablet_manager.cc:1918] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260810 00:04:05.363407 9747 log.cc:1196] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786320191480876-5521-0/minicluster-data/ts-2-root/wals/518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:05.362679 9640 raft_consensus.cc:2932] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } }
I20260810 00:04:05.364310 9502 raft_consensus.cc:2932] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } }
I20260810 00:04:05.365087 9291 catalog_manager.cc:4841] TS a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): tablet 518b732f104b4a80b1d63d0222eaf87b (table test-workload [id=5d97f0fe62ed450eafdfa0312859ddc7]) successfully deleted
I20260810 00:04:05.368779 9291 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet 518b732f104b4a80b1d63d0222eaf87b with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260810 00:04:05.373040 9302 catalog_manager.cc:5495] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d reported cstate change: config changed from index 2 to 3, VOTER a3f1f167658f46c49b5a002d88d7348c (127.5.100.67) added. New cstate: current_term: 1 leader_uuid: "1c98c733670540aeaac8ec6e489bd61d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } } }
I20260810 00:04:05.683799 9405 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:05.693037 9574 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:05.694917 9492 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:05.815564 9779 ts_tablet_manager.cc:927] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Initiating tablet copy from peer 1c98c733670540aeaac8ec6e489bd61d (127.5.100.65:34035)
I20260810 00:04:05.822607 9779 tablet_copy_client.cc:316] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.100.65:34035
I20260810 00:04:05.843149 9431 tablet_copy_service.cc:140] P 1c98c733670540aeaac8ec6e489bd61d: Received BeginTabletCopySession request for tablet 518b732f104b4a80b1d63d0222eaf87b from peer a3f1f167658f46c49b5a002d88d7348c ({username='slave'} at 127.0.0.1:35380)
I20260810 00:04:05.843708 9431 tablet_copy_service.cc:161] P 1c98c733670540aeaac8ec6e489bd61d: Beginning new tablet copy session on tablet 518b732f104b4a80b1d63d0222eaf87b from peer a3f1f167658f46c49b5a002d88d7348c at {username='slave'} at 127.0.0.1:35380: session id = a3f1f167658f46c49b5a002d88d7348c-518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:05.851313 9431 tablet_copy_source_session.cc:205] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:04:05.853492 9779 ts_tablet_manager.cc:1905] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260810 00:04:05.863750 9779 ts_tablet_manager.cc:1918] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260810 00:04:05.864353 9779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518b732f104b4a80b1d63d0222eaf87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:05.869112 9779 tablet_copy_client.cc:791] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet copy: Starting download of 0 data blocks...
I20260810 00:04:05.869632 9779 tablet_copy_client.cc:655] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet copy: Starting download of 1 WAL segments...
I20260810 00:04:05.873128 9779 tablet_copy_client.cc:531] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
ce3398b457924d618a4c44fed9b4ffc8 | 127.5.100.126:39657 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c98c733670540aeaac8ec6e489bd61d | 127.5.100.65:34035 | HEALTHY | <none> | 1 | 0
a3f1f167658f46c49b5a002d88d7348c | 127.5.100.67:43827 | HEALTHY | <none> | 0 | 0
bb54794c45954644b6c8ac8f6d388d5c | 127.5.100.66:38173 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
I20260810 00:04:05.880578 9779 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Bootstrap starting.
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 518b732f104b4a80b1d63d0222eaf87b of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
1c98c733670540aeaac8ec6e489bd61d (127.5.100.65:34035): RUNNING [LEADER]
bb54794c45954644b6c8ac8f6d388d5c (127.5.100.66:38173): RUNNING
a3f1f167658f46c49b5a002d88d7348c (127.5.100.67:43827): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = 1c98c733670540aeaac8ec6e489bd61d
B = bb54794c45954644b6c8ac8f6d388d5c
C = a3f1f167658f46c49b5a002d88d7348c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 3 | Yes
B | A* B C | 1 | 3 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:05.888471 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:05.902041 9779 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:04:05.902674 9779 tablet_bootstrap.cc:492] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Bootstrap complete.
I20260810 00:04:05.903260 9779 ts_tablet_manager.cc:1397] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.001s
I20260810 00:04:05.905184 9779 raft_consensus.cc:348] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } }
I20260810 00:04:05.905718 9779 raft_consensus.cc:729] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f1f167658f46c49b5a002d88d7348c, State: Initialized, Role: FOLLOWER
I20260810 00:04:05.906555 9779 consensus_queue.cc:260] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1c98c733670540aeaac8ec6e489bd61d" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 34035 } } peers { permanent_uuid: "bb54794c45954644b6c8ac8f6d388d5c" member_type: VOTER last_known_addr { host: "127.5.100.66" port: 38173 } } peers { permanent_uuid: "a3f1f167658f46c49b5a002d88d7348c" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 43827 } }
I20260810 00:04:05.908663 9779 ts_tablet_manager.cc:1428] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260810 00:04:05.910140 9431 tablet_copy_service.cc:342] P 1c98c733670540aeaac8ec6e489bd61d: Request end of tablet copy session a3f1f167658f46c49b5a002d88d7348c-518b732f104b4a80b1d63d0222eaf87b received from {username='slave'} at 127.0.0.1:35380
I20260810 00:04:05.910477 9431 tablet_copy_service.cc:433] P 1c98c733670540aeaac8ec6e489bd61d: ending tablet copy session a3f1f167658f46c49b5a002d88d7348c-518b732f104b4a80b1d63d0222eaf87b on tablet 518b732f104b4a80b1d63d0222eaf87b with peer a3f1f167658f46c49b5a002d88d7348c
I20260810 00:04:06.184063 9585 raft_consensus.cc:1203] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3] Dedup: 1.3->[]
I20260810 00:04:06.332559 9405 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:06.341573 9574 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:06.347947 9492 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
ce3398b457924d618a4c44fed9b4ffc8 | 127.5.100.126:39657 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c98c733670540aeaac8ec6e489bd61d | 127.5.100.65:34035 | HEALTHY | <none> | 1 | 0
a3f1f167658f46c49b5a002d88d7348c | 127.5.100.67:43827 | HEALTHY | <none> | 0 | 0
bb54794c45954644b6c8ac8f6d388d5c | 127.5.100.66:38173 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260810 00:04:06.518122 5521 log_verifier.cc:126] Checking tablet 518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:06.571666 5521 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 518b732f104b4a80b1d63d0222eaf87b
I20260810 00:04:06.574380 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:06.594838 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:06.595530 5521 tablet_replica.cc:330] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d: stopping tablet replica
I20260810 00:04:06.596091 5521 raft_consensus.cc:2229] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:06.597002 5521 raft_consensus.cc:2258] T 518b732f104b4a80b1d63d0222eaf87b P 1c98c733670540aeaac8ec6e489bd61d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:06.616293 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:06.625037 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:04:06.642649 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:06.643148 5521 tablet_replica.cc:330] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c: stopping tablet replica
I20260810 00:04:06.643627 5521 raft_consensus.cc:2229] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:04:06.644045 5521 raft_consensus.cc:2258] T 518b732f104b4a80b1d63d0222eaf87b P bb54794c45954644b6c8ac8f6d388d5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:06.662688 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:04:06.670840 5521 tablet_server.cc:178] TabletServer@127.5.100.67:0 shutting down...
I20260810 00:04:06.688190 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:06.688699 5521 tablet_replica.cc:330] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c: stopping tablet replica
I20260810 00:04:06.689136 5521 raft_consensus.cc:2229] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 00:04:06.689493 5521 raft_consensus.cc:2258] T 518b732f104b4a80b1d63d0222eaf87b P a3f1f167658f46c49b5a002d88d7348c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:06.708622 5521 tablet_server.cc:194] TabletServer@127.5.100.67:0 shutdown complete.
I20260810 00:04:06.716988 5521 master.cc:537] Master@127.5.100.126:39657 shutting down...
I20260810 00:04:06.732966 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:06.733435 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:06.733728 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P ce3398b457924d618a4c44fed9b4ffc8: stopping tablet replica
I20260810 00:04:06.751268 5521 master.cc:552] Master@127.5.100.126:39657 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (3711 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260810 00:04:06.775840 5521 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.100.126:33397
I20260810 00:04:06.776836 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:06.781379 9817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:06.782706 9818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:06.784521 9820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:06.784778 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:06.785550 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:06.785732 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:06.785844 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320246785836 us; error 0 us; skew 500 ppm
I20260810 00:04:06.786290 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:06.788591 5521 webserver.cc:460] Webserver started at http://127.5.100.126:40099/ using document root <none> and password file <none>
I20260810 00:04:06.789026 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:06.789179 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:06.789386 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:06.790419 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/master-0-root/instance:
uuid: "13ad1355261e48edbd531d6ff69601ab"
format_stamp: "Formatted at 2026-08-10 00:04:06 on dist-test-slave-8w9v"
I20260810 00:04:06.794739 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 00:04:06.797910 9826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:06.798669 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:04:06.798962 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/master-0-root
uuid: "13ad1355261e48edbd531d6ff69601ab"
format_stamp: "Formatted at 2026-08-10 00:04:06 on dist-test-slave-8w9v"
I20260810 00:04:06.799265 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:06.825752 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:06.826717 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:06.861177 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.126:33397
I20260810 00:04:06.861271 9882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.126:33397 every 8 connection(s)
I20260810 00:04:06.864980 9883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:06.875435 9883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab: Bootstrap starting.
I20260810 00:04:06.879410 9883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:06.883066 9883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab: No bootstrap required, opened a new log
I20260810 00:04:06.884845 9883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER }
I20260810 00:04:06.885192 9883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:06.885365 9883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ad1355261e48edbd531d6ff69601ab, State: Initialized, Role: FOLLOWER
I20260810 00:04:06.885888 9883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER }
I20260810 00:04:06.886304 9883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 00:04:06.886482 9883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 00:04:06.886691 9883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:06.890727 9883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER }
I20260810 00:04:06.891151 9883 leader_election.cc:302] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ad1355261e48edbd531d6ff69601ab; no voters:
I20260810 00:04:06.892218 9883 leader_election.cc:288] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 00:04:06.892565 9886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:06.893930 9886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 1 LEADER]: Becoming Leader. State: Replica: 13ad1355261e48edbd531d6ff69601ab, State: Running, Role: LEADER
I20260810 00:04:06.894575 9886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER }
I20260810 00:04:06.895129 9883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 00:04:06.896870 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13ad1355261e48edbd531d6ff69601ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER } }
I20260810 00:04:06.897301 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:06.897950 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ad1355261e48edbd531d6ff69601ab. Latest consensus state: current_term: 1 leader_uuid: "13ad1355261e48edbd531d6ff69601ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ad1355261e48edbd531d6ff69601ab" member_type: VOTER } }
I20260810 00:04:06.899305 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [sys.catalog]: This master's current role is: LEADER
I20260810 00:04:06.904687 9891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 00:04:06.911725 9891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 00:04:06.912669 5521 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260810 00:04:06.919764 9891 catalog_manager.cc:1347] Generated new cluster ID: cb3356de85874f83be2f99943b311dd1
I20260810 00:04:06.920018 9891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 00:04:06.936189 9891 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 00:04:06.937798 9891 catalog_manager.cc:1504] Loading token signing keys...
I20260810 00:04:06.948964 9891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab: Generated new TSK 0
I20260810 00:04:06.949537 9891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 00:04:06.978791 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:06.984263 9906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:06.985493 9907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:06.986892 5521 server_base.cc:1034] running on GCE node
W20260810 00:04:06.986826 9909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:06.987901 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:06.988101 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:06.988274 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320246988263 us; error 0 us; skew 500 ppm
I20260810 00:04:06.988715 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:06.990837 5521 webserver.cc:460] Webserver started at http://127.5.100.65:44417/ using document root <none> and password file <none>
I20260810 00:04:06.991235 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:06.991386 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:06.991590 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:06.992695 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root/instance:
uuid: "2ae6ad85172e4a7a930641fd778e16d0"
format_stamp: "Formatted at 2026-08-10 00:04:06 on dist-test-slave-8w9v"
I20260810 00:04:06.997064 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 00:04:07.000101 9915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.000842 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:04:07.001128 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root
uuid: "2ae6ad85172e4a7a930641fd778e16d0"
format_stamp: "Formatted at 2026-08-10 00:04:06 on dist-test-slave-8w9v"
I20260810 00:04:07.001420 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:07.019662 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:07.020743 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:07.021998 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:07.024006 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:07.024210 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.024426 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:07.024577 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.061519 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.65:33109
I20260810 00:04:07.061646 9986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.65:33109 every 8 connection(s)
I20260810 00:04:07.066210 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:07.072989 9991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:07.074438 9992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:07.078136 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:07.078418 5521 server_base.cc:1034] running on GCE node
I20260810 00:04:07.079382 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:07.079638 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:07.079875 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320247079861 us; error 0 us; skew 500 ppm
I20260810 00:04:07.080474 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:07.083176 9987 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33397
I20260810 00:04:07.083220 5521 webserver.cc:460] Webserver started at http://127.5.100.66:42001/ using document root <none> and password file <none>
I20260810 00:04:07.083722 9987 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:07.083967 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:07.084224 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:07.084492 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:07.084493 9987 heartbeater.cc:511] Master 127.5.100.126:33397 requested a full tablet report, sending...
I20260810 00:04:07.085781 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-1-root/instance:
uuid: "2b246b3d9ae84dfbb5e396352e834fff"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.086711 9844 ts_manager.cc:194] Registered new tserver with Master: 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109)
I20260810 00:04:07.088675 9844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55552
I20260810 00:04:07.090680 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:04:07.093987 10001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.094717 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 00:04:07.095057 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-1-root
uuid: "2b246b3d9ae84dfbb5e396352e834fff"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.095351 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:07.110045 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:07.110988 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:07.112288 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:07.114311 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:07.114491 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.114722 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:07.114866 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.151362 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.66:43729
I20260810 00:04:07.151531 10070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.66:43729 every 8 connection(s)
I20260810 00:04:07.156081 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:07.163043 10074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:07.164184 10075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:07.167490 5521 server_base.cc:1034] running on GCE node
W20260810 00:04:07.167512 10077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:07.168901 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:07.169193 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:07.169415 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320247169402 us; error 0 us; skew 500 ppm
I20260810 00:04:07.170083 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:07.170475 10071 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33397
I20260810 00:04:07.170926 10071 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:07.171756 10071 heartbeater.cc:511] Master 127.5.100.126:33397 requested a full tablet report, sending...
I20260810 00:04:07.172825 5521 webserver.cc:460] Webserver started at http://127.5.100.67:45913/ using document root <none> and password file <none>
I20260810 00:04:07.173481 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:07.173713 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:07.173925 9844 ts_manager.cc:194] Registered new tserver with Master: 2b246b3d9ae84dfbb5e396352e834fff (127.5.100.66:43729)
I20260810 00:04:07.174039 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:07.175580 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-2-root/instance:
uuid: "be11577a5d05412395984250550f6567"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.176162 9844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55566
I20260810 00:04:07.180240 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 00:04:07.183426 10082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.184167 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 00:04:07.184424 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-2-root
uuid: "be11577a5d05412395984250550f6567"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.184648 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:07.205739 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:07.206712 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:07.208057 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:07.210196 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:07.210361 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.210623 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:07.210775 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.247036 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.67:42601
I20260810 00:04:07.247143 10155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.67:42601 every 8 connection(s)
I20260810 00:04:07.253067 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 00:04:07.259056 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 00:04:07.263453 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:07.264838 5521 server_base.cc:1034] running on GCE node
W20260810 00:04:07.266014 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 00:04:07.267071 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260810 00:04:07.267341 5521 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260810 00:04:07.267535 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786320247267521 us; error 0 us; skew 500 ppm
I20260810 00:04:07.267899 10156 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33397
I20260810 00:04:07.268235 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 00:04:07.268302 10156 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:07.269289 10156 heartbeater.cc:511] Master 127.5.100.126:33397 requested a full tablet report, sending...
I20260810 00:04:07.271327 9844 ts_manager.cc:194] Registered new tserver with Master: be11577a5d05412395984250550f6567 (127.5.100.67:42601)
I20260810 00:04:07.271339 5521 webserver.cc:460] Webserver started at http://127.5.100.68:46779/ using document root <none> and password file <none>
I20260810 00:04:07.272092 5521 fs_manager.cc:359] Metadata directory not provided
I20260810 00:04:07.272359 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 00:04:07.272643 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 00:04:07.273221 9844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55576
I20260810 00:04:07.273811 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-3-root/instance:
uuid: "f08fe10d2bab4724a9c8aece6eda1275"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.278126 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 00:04:07.281061 10167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.281775 5521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 00:04:07.282050 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-3-root
uuid: "f08fe10d2bab4724a9c8aece6eda1275"
format_stamp: "Formatted at 2026-08-10 00:04:07 on dist-test-slave-8w9v"
I20260810 00:04:07.282308 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 00:04:07.301535 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 00:04:07.302579 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 00:04:07.303776 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 00:04:07.306161 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 00:04:07.306331 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.306568 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 00:04:07.306725 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 00:04:07.343001 5521 rpc_server.cc:307] RPC server started. Bound to: 127.5.100.68:36461
I20260810 00:04:07.343091 10239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.100.68:36461 every 8 connection(s)
I20260810 00:04:07.356738 10240 heartbeater.cc:347] Connected to a master server at 127.5.100.126:33397
I20260810 00:04:07.357100 10240 heartbeater.cc:464] Registering TS with master...
I20260810 00:04:07.357668 10240 heartbeater.cc:511] Master 127.5.100.126:33397 requested a full tablet report, sending...
I20260810 00:04:07.359390 9844 ts_manager.cc:194] Registered new tserver with Master: f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461)
I20260810 00:04:07.359573 5521 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.013673685s
I20260810 00:04:07.360978 9844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55588
I20260810 00:04:07.378958 5521 test_util.cc:268] Using random seed: 399299577
I20260810 00:04:07.400923 9844 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55608:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 00:04:07.444774 10197 tablet_service.cc:1467] Processing CreateTablet for tablet 584530e312a545b2a95e979e8e8f9585 (DEFAULT_TABLE table=test-workload [id=8a32c4eef6e243b29774cb487f13425e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:07.446027 10197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584530e312a545b2a95e979e8e8f9585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:07.453514 10115 tablet_service.cc:1467] Processing CreateTablet for tablet 584530e312a545b2a95e979e8e8f9585 (DEFAULT_TABLE table=test-workload [id=8a32c4eef6e243b29774cb487f13425e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:07.454541 10115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584530e312a545b2a95e979e8e8f9585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:07.457026 9944 tablet_service.cc:1467] Processing CreateTablet for tablet 584530e312a545b2a95e979e8e8f9585 (DEFAULT_TABLE table=test-workload [id=8a32c4eef6e243b29774cb487f13425e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 00:04:07.458214 9944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584530e312a545b2a95e979e8e8f9585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:07.462241 10263 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: Bootstrap starting.
I20260810 00:04:07.467809 10264 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: Bootstrap starting.
I20260810 00:04:07.469758 10263 tablet_bootstrap.cc:654] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:07.474143 10264 tablet_bootstrap.cc:654] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:07.478462 10265 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Bootstrap starting.
I20260810 00:04:07.480015 10263 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: No bootstrap required, opened a new log
I20260810 00:04:07.480576 10263 ts_tablet_manager.cc:1397] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.005s
I20260810 00:04:07.482438 10264 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: No bootstrap required, opened a new log
I20260810 00:04:07.482815 10264 ts_tablet_manager.cc:1397] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260810 00:04:07.483521 10263 raft_consensus.cc:348] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.484351 10263 raft_consensus.cc:374] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:07.484616 10263 raft_consensus.cc:729] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08fe10d2bab4724a9c8aece6eda1275, State: Initialized, Role: FOLLOWER
I20260810 00:04:07.485244 10265 tablet_bootstrap.cc:654] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Neither blocks nor log segments found. Creating new log.
I20260810 00:04:07.485316 10264 raft_consensus.cc:348] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.485479 10263 consensus_queue.cc:260] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.486102 10264 raft_consensus.cc:374] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:07.486547 10264 raft_consensus.cc:729] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be11577a5d05412395984250550f6567, State: Initialized, Role: FOLLOWER
I20260810 00:04:07.487219 10264 consensus_queue.cc:260] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.489437 10240 heartbeater.cc:503] Master 127.5.100.126:33397 was elected leader, sending a full tablet report...
I20260810 00:04:07.489807 10263 ts_tablet_manager.cc:1428] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260810 00:04:07.493810 10156 heartbeater.cc:503] Master 127.5.100.126:33397 was elected leader, sending a full tablet report...
I20260810 00:04:07.494180 10264 ts_tablet_manager.cc:1428] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: Time spent starting tablet: real 0.011s user 0.001s sys 0.009s
I20260810 00:04:07.497143 10265 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: No bootstrap required, opened a new log
I20260810 00:04:07.497589 10265 ts_tablet_manager.cc:1397] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20260810 00:04:07.500404 10265 raft_consensus.cc:348] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.501016 10265 raft_consensus.cc:374] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 00:04:07.501274 10265 raft_consensus.cc:729] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae6ad85172e4a7a930641fd778e16d0, State: Initialized, Role: FOLLOWER
I20260810 00:04:07.501955 10265 consensus_queue.cc:260] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.504034 9987 heartbeater.cc:503] Master 127.5.100.126:33397 was elected leader, sending a full tablet report...
I20260810 00:04:07.504532 10265 ts_tablet_manager.cc:1428] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 00:04:07.587466 10271 raft_consensus.cc:482] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 00:04:07.587939 10271 raft_consensus.cc:504] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.589773 10271 leader_election.cc:288] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461), be11577a5d05412395984250550f6567 (127.5.100.67:42601)
I20260810 00:04:07.604059 10209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "2ae6ad85172e4a7a930641fd778e16d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08fe10d2bab4724a9c8aece6eda1275" is_pre_election: true
I20260810 00:04:07.604784 10209 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ae6ad85172e4a7a930641fd778e16d0 in term 0.
I20260810 00:04:07.605716 9919 leader_election.cc:302] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ae6ad85172e4a7a930641fd778e16d0, f08fe10d2bab4724a9c8aece6eda1275; no voters:
I20260810 00:04:07.606508 10271 raft_consensus.cc:2781] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 00:04:07.606848 10271 raft_consensus.cc:482] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 00:04:07.606616 10127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "2ae6ad85172e4a7a930641fd778e16d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be11577a5d05412395984250550f6567" is_pre_election: true
I20260810 00:04:07.607161 10271 raft_consensus.cc:3037] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:07.607338 10127 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ae6ad85172e4a7a930641fd778e16d0 in term 0.
I20260810 00:04:07.611793 10271 raft_consensus.cc:504] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.613289 10271 leader_election.cc:288] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [CANDIDATE]: Term 1 election: Requested vote from peers f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461), be11577a5d05412395984250550f6567 (127.5.100.67:42601)
I20260810 00:04:07.614080 10209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "2ae6ad85172e4a7a930641fd778e16d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08fe10d2bab4724a9c8aece6eda1275"
I20260810 00:04:07.614187 10127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "2ae6ad85172e4a7a930641fd778e16d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be11577a5d05412395984250550f6567"
I20260810 00:04:07.614560 10209 raft_consensus.cc:3037] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:07.614698 10127 raft_consensus.cc:3037] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 0 FOLLOWER]: Advancing to term 1
I20260810 00:04:07.619240 10209 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ae6ad85172e4a7a930641fd778e16d0 in term 1.
I20260810 00:04:07.619233 10127 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ae6ad85172e4a7a930641fd778e16d0 in term 1.
I20260810 00:04:07.620515 9918 leader_election.cc:302] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ae6ad85172e4a7a930641fd778e16d0, be11577a5d05412395984250550f6567; no voters:
I20260810 00:04:07.621186 10271 raft_consensus.cc:2781] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 00:04:07.622027 10271 raft_consensus.cc:686] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2ae6ad85172e4a7a930641fd778e16d0, State: Running, Role: LEADER
I20260810 00:04:07.622655 10271 consensus_queue.cc:237] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:07.628897 9844 catalog_manager.cc:5495] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65). New cstate: current_term: 1 leader_uuid: "2ae6ad85172e4a7a930641fd778e16d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } health_report { overall_health: UNKNOWN } } }
I20260810 00:04:07.863307 9944 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:07.866953 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:07.873067 10197 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:07.876385 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:08.019600 10271 consensus_queue.cc:1035] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 00:04:08.045200 10276 consensus_queue.cc:1035] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 1 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 584530e312a545b2a95e979e8e8f9585 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461): RUNNING
2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): RUNNING [LEADER]
be11577a5d05412395984250550f6567 (127.5.100.67:42601): RUNNING
All reported replicas are:
A = f08fe10d2bab4724a9c8aece6eda1275
B = 2ae6ad85172e4a7a930641fd778e16d0
C = be11577a5d05412395984250550f6567
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:08.143961 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:08.179009 10071 heartbeater.cc:503] Master 127.5.100.126:33397 was elected leader, sending a full tablet report...
I20260810 00:04:08.511967 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:08.519340 9944 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:08.520753 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:08.521313 10197 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 1 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260810 00:04:08.754215 5521 log_verifier.cc:126] Checking tablet 584530e312a545b2a95e979e8e8f9585
I20260810 00:04:08.812543 5521 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 584530e312a545b2a95e979e8e8f9585
I20260810 00:04:08.813057 5521 tablet_replica.cc:330] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: stopping tablet replica
I20260810 00:04:08.813585 5521 raft_consensus.cc:2229] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:08.814399 5521 raft_consensus.cc:2258] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:09.036926 9944 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.050593 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.053015 10197 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.059522 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 0 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 584530e312a545b2a95e979e8e8f9585 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461): RUNNING
2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
be11577a5d05412395984250550f6567 (127.5.100.67:42601): RUNNING
All reported replicas are:
A = f08fe10d2bab4724a9c8aece6eda1275
B = 2ae6ad85172e4a7a930641fd778e16d0
C = be11577a5d05412395984250550f6567
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:09.278491 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:09.636562 9944 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.648520 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.654000 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:09.656320 10197 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 0 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 584530e312a545b2a95e979e8e8f9585 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461): RUNNING
2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
be11577a5d05412395984250550f6567 (127.5.100.67:42601): RUNNING
All reported replicas are:
A = f08fe10d2bab4724a9c8aece6eda1275
B = 2ae6ad85172e4a7a930641fd778e16d0
C = be11577a5d05412395984250550f6567
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:09.864054 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:10.016052 10415 raft_consensus.cc:482] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ae6ad85172e4a7a930641fd778e16d0)
I20260810 00:04:10.016589 10415 raft_consensus.cc:504] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:10.018684 10415 leader_election.cc:288] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461), 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109)
I20260810 00:04:10.033420 10209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "be11577a5d05412395984250550f6567" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f08fe10d2bab4724a9c8aece6eda1275" is_pre_election: true
I20260810 00:04:10.034016 9956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "be11577a5d05412395984250550f6567" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ae6ad85172e4a7a930641fd778e16d0" is_pre_election: true
W20260810 00:04:10.035131 10086 leader_election.cc:341] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): Illegal state: cannot vote while shut down
I20260810 00:04:10.035507 10086 leader_election.cc:302] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be11577a5d05412395984250550f6567; no voters: 2ae6ad85172e4a7a930641fd778e16d0, f08fe10d2bab4724a9c8aece6eda1275
I20260810 00:04:10.036259 10415 raft_consensus.cc:2726] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 00:04:10.125732 10427 raft_consensus.cc:482] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ae6ad85172e4a7a930641fd778e16d0)
I20260810 00:04:10.126202 10427 raft_consensus.cc:504] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:10.128226 10427 leader_election.cc:288] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109), be11577a5d05412395984250550f6567 (127.5.100.67:42601)
I20260810 00:04:10.143080 10127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "f08fe10d2bab4724a9c8aece6eda1275" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be11577a5d05412395984250550f6567" is_pre_election: true
I20260810 00:04:10.143669 10127 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08fe10d2bab4724a9c8aece6eda1275 in term 1.
I20260810 00:04:10.144694 10170 leader_election.cc:302] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be11577a5d05412395984250550f6567, f08fe10d2bab4724a9c8aece6eda1275; no voters:
I20260810 00:04:10.145515 10427 raft_consensus.cc:2781] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 00:04:10.145597 9956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "f08fe10d2bab4724a9c8aece6eda1275" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ae6ad85172e4a7a930641fd778e16d0" is_pre_election: true
I20260810 00:04:10.145877 10427 raft_consensus.cc:482] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ae6ad85172e4a7a930641fd778e16d0)
I20260810 00:04:10.146392 10427 raft_consensus.cc:3037] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 1 FOLLOWER]: Advancing to term 2
W20260810 00:04:10.147202 10171 leader_election.cc:341] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): Illegal state: cannot vote while shut down
I20260810 00:04:10.151557 10427 raft_consensus.cc:504] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:10.153209 10427 leader_election.cc:288] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 election: Requested vote from peers 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109), be11577a5d05412395984250550f6567 (127.5.100.67:42601)
I20260810 00:04:10.153891 9956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "f08fe10d2bab4724a9c8aece6eda1275" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ae6ad85172e4a7a930641fd778e16d0"
I20260810 00:04:10.154192 10127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "584530e312a545b2a95e979e8e8f9585" candidate_uuid: "f08fe10d2bab4724a9c8aece6eda1275" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be11577a5d05412395984250550f6567"
I20260810 00:04:10.154685 10127 raft_consensus.cc:3037] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 1 FOLLOWER]: Advancing to term 2
W20260810 00:04:10.154956 10171 leader_election.cc:341] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): Illegal state: cannot vote while shut down
I20260810 00:04:10.159135 10127 raft_consensus.cc:2454] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08fe10d2bab4724a9c8aece6eda1275 in term 2.
I20260810 00:04:10.160025 10170 leader_election.cc:302] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be11577a5d05412395984250550f6567, f08fe10d2bab4724a9c8aece6eda1275; no voters: 2ae6ad85172e4a7a930641fd778e16d0
I20260810 00:04:10.160732 10427 raft_consensus.cc:2781] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 00:04:10.161639 10427 raft_consensus.cc:686] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 LEADER]: Becoming Leader. State: Replica: f08fe10d2bab4724a9c8aece6eda1275, State: Running, Role: LEADER
I20260810 00:04:10.162528 10427 consensus_queue.cc:237] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:10.169200 9844 catalog_manager.cc:5495] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 reported cstate change: term changed from 1 to 2, leader changed from 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65) to f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68). New cstate: current_term: 2 leader_uuid: "f08fe10d2bab4724a9c8aece6eda1275" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ae6ad85172e4a7a930641fd778e16d0" member_type: VOTER last_known_addr { host: "127.5.100.65" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } health_report { overall_health: UNKNOWN } } }
I20260810 00:04:10.310338 9944 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:10.318285 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:10.324227 10197 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:10.325445 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 0 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 584530e312a545b2a95e979e8e8f9585 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461): RUNNING [LEADER]
2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
be11577a5d05412395984250550f6567 (127.5.100.67:42601): RUNNING
All reported replicas are:
A = f08fe10d2bab4724a9c8aece6eda1275
B = 2ae6ad85172e4a7a930641fd778e16d0
C = be11577a5d05412395984250550f6567
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | -1 | Yes
B | [config not available] | | |
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260810 00:04:10.521566 5521 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260810 00:04:10.611066 10171 consensus_queue.cc:579] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: The tablet replica hosted on peer 2ae6ad85172e4a7a930641fd778e16d0 has failed
W20260810 00:04:10.611545 10171 consensus_peers.cc:487] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 -> Peer 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): Couldn't send request to peer 2ae6ad85172e4a7a930641fd778e16d0. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:04:10.669054 10127 raft_consensus.cc:1261] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Refusing update from remote peer f08fe10d2bab4724a9c8aece6eda1275: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260810 00:04:10.670235 10430 consensus_queue.cc:1035] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260810 00:04:10.685685 10209 consensus_queue.cc:237] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } }
I20260810 00:04:10.689191 10127 raft_consensus.cc:1261] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Refusing update from remote peer f08fe10d2bab4724a9c8aece6eda1275: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260810 00:04:10.690202 10430 consensus_queue.cc:1035] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 00:04:10.694821 10427 raft_consensus.cc:2932] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } }
I20260810 00:04:10.696303 10127 raft_consensus.cc:2932] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } }
I20260810 00:04:10.702080 9830 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 584530e312a545b2a95e979e8e8f9585 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260810 00:04:10.705703 9844 catalog_manager.cc:5495] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 reported cstate change: config changed from index -1 to 3, VOTER 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65) evicted. New cstate: current_term: 2 leader_uuid: "f08fe10d2bab4724a9c8aece6eda1275" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } health_report { overall_health: HEALTHY } } }
I20260810 00:04:10.715490 9944 tablet_service.cc:1514] Processing DeleteTablet for tablet 584530e312a545b2a95e979e8e8f9585 with delete_type TABLET_DATA_TOMBSTONED (TS 2ae6ad85172e4a7a930641fd778e16d0 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:46210
I20260810 00:04:10.717578 10466 ts_tablet_manager.cc:1905] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260810 00:04:10.718822 10209 consensus_queue.cc:237] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } }
I20260810 00:04:10.725383 10127 raft_consensus.cc:1261] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Refusing update from remote peer f08fe10d2bab4724a9c8aece6eda1275: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260810 00:04:10.726907 10427 consensus_queue.cc:1035] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260810 00:04:10.731670 10466 ts_tablet_manager.cc:1918] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260810 00:04:10.732079 10466 log.cc:1196] T 584530e312a545b2a95e979e8e8f9585 P 2ae6ad85172e4a7a930641fd778e16d0: Deleting WAL directory at /tmp/dist-test-taskeau3yT/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786320191480876-5521-0/minicluster-data/ts-0-root/wals/584530e312a545b2a95e979e8e8f9585
I20260810 00:04:10.732455 10430 raft_consensus.cc:2932] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 2b246b3d9ae84dfbb5e396352e834fff (127.5.100.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } } }
I20260810 00:04:10.733827 9830 catalog_manager.cc:4841] TS 2ae6ad85172e4a7a930641fd778e16d0 (127.5.100.65:33109): tablet 584530e312a545b2a95e979e8e8f9585 (table test-workload [id=8a32c4eef6e243b29774cb487f13425e]) successfully deleted
I20260810 00:04:10.735186 10127 raft_consensus.cc:2932] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 2b246b3d9ae84dfbb5e396352e834fff (127.5.100.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } } }
W20260810 00:04:10.737480 10170 consensus_peers.cc:487] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 -> Peer 2b246b3d9ae84dfbb5e396352e834fff (127.5.100.66:43729): Couldn't send request to peer 2b246b3d9ae84dfbb5e396352e834fff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 584530e312a545b2a95e979e8e8f9585. This is attempt 1: this message will repeat every 5th retry.
I20260810 00:04:10.739843 9830 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 584530e312a545b2a95e979e8e8f9585 with cas_config_opid_index 3: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260810 00:04:10.743629 9844 catalog_manager.cc:5495] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 reported cstate change: config changed from index 3 to 4, NON_VOTER 2b246b3d9ae84dfbb5e396352e834fff (127.5.100.66) added. New cstate: current_term: 2 leader_uuid: "f08fe10d2bab4724a9c8aece6eda1275" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260810 00:04:11.071065 10029 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:11.079978 9944 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:11.088088 10197 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260810 00:04:11.093911 10115 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260810 00:04:11.127635 10502 ts_tablet_manager.cc:927] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Initiating tablet copy from peer f08fe10d2bab4724a9c8aece6eda1275 (127.5.100.68:36461)
I20260810 00:04:11.152954 10502 tablet_copy_client.cc:316] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: tablet copy: Beginning tablet copy session from remote peer at address 127.5.100.68:36461
I20260810 00:04:11.202224 10222 tablet_copy_service.cc:140] P f08fe10d2bab4724a9c8aece6eda1275: Received BeginTabletCopySession request for tablet 584530e312a545b2a95e979e8e8f9585 from peer 2b246b3d9ae84dfbb5e396352e834fff ({username='slave'} at 127.0.0.1:42514)
I20260810 00:04:11.202852 10222 tablet_copy_service.cc:161] P f08fe10d2bab4724a9c8aece6eda1275: Beginning new tablet copy session on tablet 584530e312a545b2a95e979e8e8f9585 from peer 2b246b3d9ae84dfbb5e396352e834fff at {username='slave'} at 127.0.0.1:42514: session id = 2b246b3d9ae84dfbb5e396352e834fff-584530e312a545b2a95e979e8e8f9585
I20260810 00:04:11.211277 10222 tablet_copy_source_session.cc:205] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 00:04:11.214522 10502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584530e312a545b2a95e979e8e8f9585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 00:04:11.236690 10502 tablet_copy_client.cc:791] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: tablet copy: Starting download of 0 data blocks...
I20260810 00:04:11.237303 10502 tablet_copy_client.cc:655] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: tablet copy: Starting download of 1 WAL segments...
I20260810 00:04:11.241251 10502 tablet_copy_client.cc:531] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 00:04:11.256718 10502 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Bootstrap starting.
I20260810 00:04:11.295564 10502 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 00:04:11.296561 10502 tablet_bootstrap.cc:492] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Bootstrap complete.
I20260810 00:04:11.297251 10502 ts_tablet_manager.cc:1397] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.021s
I20260810 00:04:11.300060 10502 raft_consensus.cc:348] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } }
I20260810 00:04:11.300951 10502 raft_consensus.cc:729] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2b246b3d9ae84dfbb5e396352e834fff, State: Initialized, Role: LEARNER
I20260810 00:04:11.301558 10502 consensus_queue.cc:260] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f08fe10d2bab4724a9c8aece6eda1275" member_type: VOTER last_known_addr { host: "127.5.100.68" port: 36461 } } peers { permanent_uuid: "be11577a5d05412395984250550f6567" member_type: VOTER last_known_addr { host: "127.5.100.67" port: 42601 } } peers { permanent_uuid: "2b246b3d9ae84dfbb5e396352e834fff" member_type: NON_VOTER last_known_addr { host: "127.5.100.66" port: 43729 } attrs { promote: true } }
I20260810 00:04:11.306185 10502 ts_tablet_manager.cc:1428] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260810 00:04:11.310526 10222 tablet_copy_service.cc:342] P f08fe10d2bab4724a9c8aece6eda1275: Request end of tablet copy session 2b246b3d9ae84dfbb5e396352e834fff-584530e312a545b2a95e979e8e8f9585 received from {username='slave'} at 127.0.0.1:42514
I20260810 00:04:11.310941 10222 tablet_copy_service.cc:433] P f08fe10d2bab4724a9c8aece6eda1275: ending tablet copy session 2b246b3d9ae84dfbb5e396352e834fff-584530e312a545b2a95e979e8e8f9585 on tablet 584530e312a545b2a95e979e8e8f9585 with peer 2b246b3d9ae84dfbb5e396352e834fff
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
13ad1355261e48edbd531d6ff69601ab | 127.5.100.126:33397 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | HEALTHY | <none> | 0 | 0
2b246b3d9ae84dfbb5e396352e834fff | 127.5.100.66:43729 | HEALTHY | <none> | 0 | 0
be11577a5d05412395984250550f6567 | 127.5.100.67:42601 | HEALTHY | <none> | 0 | 0
f08fe10d2bab4724a9c8aece6eda1275 | 127.5.100.68:36461 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 2ae6ad85172e4a7a930641fd778e16d0 | 127.5.100.65:33109 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260810 00:04:11.346117 5521 log_verifier.cc:126] Checking tablet 584530e312a545b2a95e979e8e8f9585
I20260810 00:04:11.401026 5521 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 584530e312a545b2a95e979e8e8f9585
I20260810 00:04:11.402976 5521 tablet_server.cc:178] TabletServer@127.5.100.65:0 shutting down...
I20260810 00:04:11.427016 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:11.444238 5521 tablet_server.cc:194] TabletServer@127.5.100.65:0 shutdown complete.
I20260810 00:04:11.452806 5521 tablet_server.cc:178] TabletServer@127.5.100.66:0 shutting down...
I20260810 00:04:11.471134 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:11.471793 5521 tablet_replica.cc:330] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff: stopping tablet replica
I20260810 00:04:11.472409 5521 raft_consensus.cc:2229] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff [term 2 LEARNER]: Raft consensus shutting down.
I20260810 00:04:11.472740 5521 raft_consensus.cc:2258] T 584530e312a545b2a95e979e8e8f9585 P 2b246b3d9ae84dfbb5e396352e834fff [term 2 LEARNER]: Raft consensus is shut down!
I20260810 00:04:11.491941 5521 tablet_server.cc:194] TabletServer@127.5.100.66:0 shutdown complete.
I20260810 00:04:11.501226 5521 tablet_server.cc:178] TabletServer@127.5.100.67:0 shutting down...
I20260810 00:04:11.520186 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:11.520815 5521 tablet_replica.cc:330] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567: stopping tablet replica
I20260810 00:04:11.521289 5521 raft_consensus.cc:2229] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 00:04:11.521884 5521 raft_consensus.cc:2258] T 584530e312a545b2a95e979e8e8f9585 P be11577a5d05412395984250550f6567 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:11.541424 5521 tablet_server.cc:194] TabletServer@127.5.100.67:0 shutdown complete.
I20260810 00:04:11.549716 5521 tablet_server.cc:178] TabletServer@127.5.100.68:0 shutting down...
I20260810 00:04:11.567770 5521 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260810 00:04:11.568362 5521 tablet_replica.cc:330] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275: stopping tablet replica
I20260810 00:04:11.568887 5521 raft_consensus.cc:2229] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 LEADER]: Raft consensus shutting down.
I20260810 00:04:11.569702 5521 raft_consensus.cc:2258] T 584530e312a545b2a95e979e8e8f9585 P f08fe10d2bab4724a9c8aece6eda1275 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:11.588933 5521 tablet_server.cc:194] TabletServer@127.5.100.68:0 shutdown complete.
I20260810 00:04:11.597728 5521 master.cc:537] Master@127.5.100.126:33397 shutting down...
I20260810 00:04:11.614125 5521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 1 LEADER]: Raft consensus shutting down.
I20260810 00:04:11.614593 5521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 00:04:11.614952 5521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 13ad1355261e48edbd531d6ff69601ab: stopping tablet replica
I20260810 00:04:11.632390 5521 master.cc:552] Master@127.5.100.126:33397 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (4882 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (8594 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (59968 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260810 00:04:11.665480 5521 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~9 seconds ago
I20260810 00:04:11.665800 5521 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~13 seconds ago
I20260810 00:04:11.665975 5521 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 63 messages since previous log ~51 seconds ago
I20260810 00:04:11.666285 5521 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 8 messages since previous log ~12 seconds ago