[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 05:41:41.484040 10684 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.330s user 0.000s sys 0.004s
W20260807 05:41:41.484668 10684 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.331s user 0.000s sys 0.004s
I20260807 05:41:41.489081 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:39319
I20260807 05:41:41.491581 10684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:41:41.493032 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:41.518973 10698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:41.523674 10700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:41.519448 10697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:41.690275 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:41.691617 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:41.691888 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:41.692090 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081301692066 us; error 0 us; skew 500 ppm
I20260807 05:41:41.693267 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:41.699774 10684 webserver.cc:460] Webserver started at http://127.10.111.62:46641/ using document root <none> and password file <none>
I20260807 05:41:41.701035 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:41.701272 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:41.701857 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:41.709241 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "f81dba01fd354442a74735a542535164"
format_stamp: "Formatted at 2026-08-07 05:41:41 on dist-test-slave-qg60"
I20260807 05:41:41.719503 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.008s
I20260807 05:41:41.726655 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:41.728283 10684 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.003s
I20260807 05:41:41.728843 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "f81dba01fd354442a74735a542535164"
format_stamp: "Formatted at 2026-08-07 05:41:41 on dist-test-slave-qg60"
I20260807 05:41:41.729230 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:41.749281 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:41.751134 10684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:41:41.751765 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:41.861974 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:39319
I20260807 05:41:41.862097 10758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:39319 every 8 connection(s)
I20260807 05:41:41.872016 10759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:41.903177 10759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: Bootstrap starting.
I20260807 05:41:41.912420 10759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:41.915131 10759 log.cc:824] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: Log is configured to *not* fsync() on all Append() calls
I20260807 05:41:41.921705 10759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: No bootstrap required, opened a new log
I20260807 05:41:41.950821 10759 raft_consensus.cc:348] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER }
I20260807 05:41:41.951675 10759 raft_consensus.cc:374] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:41.951941 10759 raft_consensus.cc:729] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f81dba01fd354442a74735a542535164, State: Initialized, Role: FOLLOWER
I20260807 05:41:41.953080 10759 consensus_queue.cc:260] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER }
I20260807 05:41:41.953636 10759 raft_consensus.cc:388] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:41:41.953902 10759 raft_consensus.cc:482] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:41:41.954219 10759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:41.959420 10759 raft_consensus.cc:504] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER }
I20260807 05:41:41.960423 10759 leader_election.cc:302] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f81dba01fd354442a74735a542535164; no voters:
I20260807 05:41:41.962929 10759 leader_election.cc:288] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:41:41.963253 10762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:41.967046 10762 raft_consensus.cc:686] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 1 LEADER]: Becoming Leader. State: Replica: f81dba01fd354442a74735a542535164, State: Running, Role: LEADER
I20260807 05:41:41.968238 10762 consensus_queue.cc:237] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER }
I20260807 05:41:41.969684 10759 sys_catalog.cc:564] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:41.980326 10764 sys_catalog.cc:455] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f81dba01fd354442a74735a542535164. Latest consensus state: current_term: 1 leader_uuid: "f81dba01fd354442a74735a542535164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER } }
I20260807 05:41:41.981866 10764 sys_catalog.cc:458] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:41.981571 10763 sys_catalog.cc:455] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f81dba01fd354442a74735a542535164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f81dba01fd354442a74735a542535164" member_type: VOTER } }
I20260807 05:41:41.982366 10763 sys_catalog.cc:458] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:41.988544 10773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:42.008953 10773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:42.018857 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:42.033820 10773 catalog_manager.cc:1347] Generated new cluster ID: 6fae647f70b244f8bc443edbac830cb8
I20260807 05:41:42.034164 10773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:42.075124 10773 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:42.077539 10773 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:42.100124 10773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: Generated new TSK 0
I20260807 05:41:42.101715 10773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:42.151328 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:42.165161 10783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:42.168380 10784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:42.170069 10786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:42.171706 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:42.172653 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:42.172995 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:42.173220 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081302173204 us; error 0 us; skew 500 ppm
I20260807 05:41:42.174022 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:42.179322 10684 webserver.cc:460] Webserver started at http://127.10.111.1:33719/ using document root <none> and password file <none>
I20260807 05:41:42.180147 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:42.180395 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:42.180785 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:42.182760 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "9f1d4cb4dd704374aaee70e6341fbfdd"
format_stamp: "Formatted at 2026-08-07 05:41:42 on dist-test-slave-qg60"
I20260807 05:41:42.191598 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260807 05:41:42.197271 10791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.198213 10684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 05:41:42.198580 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "9f1d4cb4dd704374aaee70e6341fbfdd"
format_stamp: "Formatted at 2026-08-07 05:41:42 on dist-test-slave-qg60"
I20260807 05:41:42.198971 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:42.217957 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:42.219980 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:42.223730 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:42.229804 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:42.230146 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.230479 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:42.230715 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.364890 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:40021
I20260807 05:41:42.365005 10858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:40021 every 8 connection(s)
I20260807 05:41:42.374105 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:42.400269 10864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:42.405836 10863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:42.420478 10859 heartbeater.cc:347] Connected to a master server at 127.10.111.62:39319
I20260807 05:41:42.421002 10859 heartbeater.cc:464] Registering TS with master...
W20260807 05:41:42.421270 10866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:42.422258 10859 heartbeater.cc:511] Master 127.10.111.62:39319 requested a full tablet report, sending...
I20260807 05:41:42.426654 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:42.426810 10722 ts_manager.cc:194] Registered new tserver with Master: 9f1d4cb4dd704374aaee70e6341fbfdd (127.10.111.1:40021)
I20260807 05:41:42.427845 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:42.428133 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:42.428369 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081302428341 us; error 0 us; skew 500 ppm
I20260807 05:41:42.429199 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:42.430846 10722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43182
I20260807 05:41:42.432869 10684 webserver.cc:460] Webserver started at http://127.10.111.2:41533/ using document root <none> and password file <none>
I20260807 05:41:42.433368 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:42.433589 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:42.433873 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:42.434993 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "c97de53a43de4f99bb1f74887936a162"
format_stamp: "Formatted at 2026-08-07 05:41:42 on dist-test-slave-qg60"
I20260807 05:41:42.440578 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 05:41:42.445470 10871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.446393 10684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:41:42.446681 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "c97de53a43de4f99bb1f74887936a162"
format_stamp: "Formatted at 2026-08-07 05:41:42 on dist-test-slave-qg60"
I20260807 05:41:42.446979 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:42.484674 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:42.486147 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:42.487752 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:42.489944 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:42.490164 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.490430 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:42.490610 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:42.536978 10859 heartbeater.cc:503] Master 127.10.111.62:39319 was elected leader, sending a full tablet report...
I20260807 05:41:42.634850 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:33981
I20260807 05:41:42.635033 10936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:33981 every 8 connection(s)
I20260807 05:41:42.663371 10937 heartbeater.cc:347] Connected to a master server at 127.10.111.62:39319
I20260807 05:41:42.663793 10937 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:42.664722 10937 heartbeater.cc:511] Master 127.10.111.62:39319 requested a full tablet report, sending...
I20260807 05:41:42.667614 10722 ts_manager.cc:194] Registered new tserver with Master: c97de53a43de4f99bb1f74887936a162 (127.10.111.2:33981)
I20260807 05:41:42.668718 10684 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.020436661s
I20260807 05:41:42.669838 10722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43198
I20260807 05:41:42.713881 10722 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:41:42.741789 10722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260807 05:41:42.774112 10937 heartbeater.cc:503] Master 127.10.111.62:39319 was elected leader, sending a full tablet report...
I20260807 05:41:42.837044 10821 tablet_service.cc:1467] Processing CreateTablet for tablet 66b2f16ab002408ab2ff098a78b386ba (DEFAULT_TABLE table=test-table [id=21f745880af246f2aed1551ddf8e3065]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:42.837847 10822 tablet_service.cc:1467] Processing CreateTablet for tablet 2381e517a1b94797ab13438ddb52c5d5 (DEFAULT_TABLE table=test-table [id=21f745880af246f2aed1551ddf8e3065]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:42.839628 10821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66b2f16ab002408ab2ff098a78b386ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:42.840662 10822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2381e517a1b94797ab13438ddb52c5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:42.840889 10900 tablet_service.cc:1467] Processing CreateTablet for tablet 2381e517a1b94797ab13438ddb52c5d5 (DEFAULT_TABLE table=test-table [id=21f745880af246f2aed1551ddf8e3065]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:42.842972 10900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2381e517a1b94797ab13438ddb52c5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:42.842865 10899 tablet_service.cc:1467] Processing CreateTablet for tablet 66b2f16ab002408ab2ff098a78b386ba (DEFAULT_TABLE table=test-table [id=21f745880af246f2aed1551ddf8e3065]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:42.844794 10899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66b2f16ab002408ab2ff098a78b386ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:42.866649 10953 tablet_bootstrap.cc:492] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: Bootstrap starting.
I20260807 05:41:42.872546 10954 tablet_bootstrap.cc:492] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: Bootstrap starting.
I20260807 05:41:42.885052 10954 tablet_bootstrap.cc:654] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:42.890558 10953 tablet_bootstrap.cc:654] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:42.894964 10954 tablet_bootstrap.cc:492] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: No bootstrap required, opened a new log
I20260807 05:41:42.895661 10954 ts_tablet_manager.cc:1397] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.014s
I20260807 05:41:42.899727 10954 raft_consensus.cc:348] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:42.900656 10954 raft_consensus.cc:729] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Initialized, Role: FOLLOWER
I20260807 05:41:42.901341 10954 consensus_queue.cc:260] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:42.904418 10953 tablet_bootstrap.cc:492] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: No bootstrap required, opened a new log
I20260807 05:41:42.904968 10953 ts_tablet_manager.cc:1397] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: Time spent bootstrapping tablet: real 0.039s user 0.013s sys 0.008s
I20260807 05:41:42.906387 10954 ts_tablet_manager.cc:1428] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260807 05:41:42.907423 10954 tablet_bootstrap.cc:492] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: Bootstrap starting.
I20260807 05:41:42.911034 10953 raft_consensus.cc:348] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:42.912271 10953 raft_consensus.cc:729] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1d4cb4dd704374aaee70e6341fbfdd, State: Initialized, Role: FOLLOWER
I20260807 05:41:42.913137 10953 consensus_queue.cc:260] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:42.917455 10954 tablet_bootstrap.cc:654] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:42.934831 10954 tablet_bootstrap.cc:492] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: No bootstrap required, opened a new log
I20260807 05:41:42.935264 10954 ts_tablet_manager.cc:1397] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: Time spent bootstrapping tablet: real 0.028s user 0.016s sys 0.004s
I20260807 05:41:42.938460 10954 raft_consensus.cc:348] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:42.939127 10954 raft_consensus.cc:729] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Initialized, Role: FOLLOWER
I20260807 05:41:42.939729 10954 consensus_queue.cc:260] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:42.942576 10953 ts_tablet_manager.cc:1428] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: Time spent starting tablet: real 0.037s user 0.007s sys 0.005s
I20260807 05:41:42.944859 10954 ts_tablet_manager.cc:1428] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: Time spent starting tablet: real 0.009s user 0.002s sys 0.007s
I20260807 05:41:42.946197 10953 tablet_bootstrap.cc:492] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: Bootstrap starting.
I20260807 05:41:42.954777 10953 tablet_bootstrap.cc:654] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:42.963100 10953 tablet_bootstrap.cc:492] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: No bootstrap required, opened a new log
I20260807 05:41:42.963511 10953 ts_tablet_manager.cc:1397] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260807 05:41:42.966158 10953 raft_consensus.cc:348] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:42.966668 10953 raft_consensus.cc:729] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1d4cb4dd704374aaee70e6341fbfdd, State: Initialized, Role: FOLLOWER
I20260807 05:41:42.967257 10953 consensus_queue.cc:260] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:42.969544 10953 ts_tablet_manager.cc:1428] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 05:41:43.023154 10684 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260807 05:41:43.023568 10684 raft_consensus.cc:412] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Emulating election...
I20260807 05:41:43.023802 10684 raft_consensus.cc:3037] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:43.028524 10684 raft_consensus.cc:686] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Becoming Leader. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:43.029384 10684 consensus_queue.cc:237] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:43.033720 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:43.042392 10722 catalog_manager.cc:5495] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97de53a43de4f99bb1f74887936a162 (127.10.111.2). New cstate: current_term: 1 leader_uuid: "c97de53a43de4f99bb1f74887936a162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:43.065824 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
W20260807 05:41:43.123852 10860 tablet.cc:2369] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:41:43.169811 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 05:41:43.374393 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 05:41:43.569950 10832 raft_consensus.cc:3037] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:43.578469 10965 consensus_queue.cc:1035] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:43.678231 10684 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 05:41:43.678542 10684 raft_consensus.cc:412] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Emulating election...
I20260807 05:41:43.678823 10684 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:43.684696 10684 raft_consensus.cc:686] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Becoming Leader. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:43.685449 10684 consensus_queue.cc:237] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:43.688144 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:43.693555 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 05:41:43.693825 10721 catalog_manager.cc:5495] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97de53a43de4f99bb1f74887936a162 (127.10.111.2). New cstate: current_term: 1 leader_uuid: "c97de53a43de4f99bb1f74887936a162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: HEALTHY } } }
I20260807 05:41:43.798121 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 05:41:44.003011 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 05:41:44.254555 10832 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:44.260015 10965 consensus_queue.cc:1035] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:44.507138 10684 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 05:41:44.507395 10684 raft_consensus.cc:412] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Emulating election...
I20260807 05:41:44.507622 10684 raft_consensus.cc:3032] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:44.507793 10684 raft_consensus.cc:729] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:44.508351 10684 consensus_queue.cc:260] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:44.509235 10684 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:44.513732 10684 raft_consensus.cc:686] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 2 LEADER]: Becoming Leader. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:44.514317 10684 consensus_queue.cc:237] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:44.515964 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:44.520121 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 05:41:44.519557 10722 catalog_manager.cc:5495] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "c97de53a43de4f99bb1f74887936a162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: HEALTHY } } }
I20260807 05:41:44.623819 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 05:41:44.827508 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 05:41:44.928850 10832 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:44.933597 10832 raft_consensus.cc:1261] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 FOLLOWER]: Refusing update from remote peer c97de53a43de4f99bb1f74887936a162: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:44.934808 10982 consensus_queue.cc:1035] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:45.132078 10684 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 05:41:45.132395 10684 raft_consensus.cc:412] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 1 FOLLOWER]: Emulating election...
I20260807 05:41:45.132752 10684 raft_consensus.cc:3037] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:45.137089 10684 raft_consensus.cc:686] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 LEADER]: Becoming Leader. State: Replica: 9f1d4cb4dd704374aaee70e6341fbfdd, State: Running, Role: LEADER
I20260807 05:41:45.137652 10684 consensus_queue.cc:237] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:45.139628 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:45.143635 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 05:41:45.143785 10722 catalog_manager.cc:5495] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd reported cstate change: term changed from 1 to 2, leader changed from c97de53a43de4f99bb1f74887936a162 (127.10.111.2) to 9f1d4cb4dd704374aaee70e6341fbfdd (127.10.111.1). New cstate: current_term: 2 leader_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: HEALTHY } } }
I20260807 05:41:45.247929 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 05:41:45.451856 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 05:41:45.588056 10912 raft_consensus.cc:3032] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:45.588557 10912 raft_consensus.cc:729] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:45.589183 10912 consensus_queue.cc:260] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } } peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } }
I20260807 05:41:45.589956 10912 raft_consensus.cc:3037] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:45.596248 10912 raft_consensus.cc:1261] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 2 FOLLOWER]: Refusing update from remote peer 9f1d4cb4dd704374aaee70e6341fbfdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:45.598097 10988 consensus_queue.cc:1035] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:45.956128 10684 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260807 05:41:45.956373 10684 raft_consensus.cc:412] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 FOLLOWER]: Emulating election...
I20260807 05:41:45.956619 10684 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:45.960597 10684 raft_consensus.cc:686] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 3 LEADER]: Becoming Leader. State: Replica: 9f1d4cb4dd704374aaee70e6341fbfdd, State: Running, Role: LEADER
I20260807 05:41:45.961319 10684 consensus_queue.cc:237] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:45.963142 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:45.967185 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 05:41:45.966753 10720 catalog_manager.cc:5495] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd reported cstate change: term changed from 2 to 3, leader changed from c97de53a43de4f99bb1f74887936a162 (127.10.111.2) to 9f1d4cb4dd704374aaee70e6341fbfdd (127.10.111.1). New cstate: current_term: 3 leader_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:46.071427 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 05:41:46.275718 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 05:41:46.485484 10832 raft_consensus.cc:1226] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 3 LEADER]: Rejecting Update request from peer c97de53a43de4f99bb1f74887936a162 for earlier term 2. Current term is 3. Ops: []
I20260807 05:41:46.486570 11007 consensus_queue.cc:1046] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 05:41:46.487646 11007 consensus_queue.cc:1225] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Peer 9f1d4cb4dd704374aaee70e6341fbfdd log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 05:41:46.488168 11009 raft_consensus.cc:3032] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 05:41:46.488389 11009 raft_consensus.cc:729] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:46.488866 11009 consensus_queue.cc:260] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:46.489611 11009 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:46.510814 10912 raft_consensus.cc:1261] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 3 FOLLOWER]: Refusing update from remote peer 9f1d4cb4dd704374aaee70e6341fbfdd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 05:41:46.512241 10988 consensus_queue.cc:1035] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 05:41:46.579810 10684 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 05:41:46.580103 10684 raft_consensus.cc:412] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 3 FOLLOWER]: Emulating election...
I20260807 05:41:46.580361 10684 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:41:46.584583 10684 raft_consensus.cc:686] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 4 LEADER]: Becoming Leader. State: Replica: c97de53a43de4f99bb1f74887936a162, State: Running, Role: LEADER
I20260807 05:41:46.585170 10684 consensus_queue.cc:237] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:46.586611 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:46.590498 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 05:41:46.590371 10722 catalog_manager.cc:5495] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 reported cstate change: term changed from 3 to 4, leader changed from 9f1d4cb4dd704374aaee70e6341fbfdd (127.10.111.1) to c97de53a43de4f99bb1f74887936a162 (127.10.111.2). New cstate: current_term: 4 leader_uuid: "c97de53a43de4f99bb1f74887936a162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } health_report { overall_health: HEALTHY } } }
I20260807 05:41:46.694464 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 05:41:46.898185 10684 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 05:41:47.021063 10912 raft_consensus.cc:1226] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 4 LEADER]: Rejecting Update request from peer 9f1d4cb4dd704374aaee70e6341fbfdd for earlier term 3. Current term is 4. Ops: []
I20260807 05:41:47.022854 10988 consensus_queue.cc:1046] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:41:47.024031 10988 consensus_queue.cc:1225] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [LEADER]: Peer c97de53a43de4f99bb1f74887936a162 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 05:41:47.024448 11019 raft_consensus.cc:3032] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 3 LEADER]: Stepping down as leader of term 3
I20260807 05:41:47.024793 11019 raft_consensus.cc:729] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9f1d4cb4dd704374aaee70e6341fbfdd, State: Running, Role: LEADER
I20260807 05:41:47.025401 11019 consensus_queue.cc:260] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 } } peers { permanent_uuid: "c97de53a43de4f99bb1f74887936a162" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 33981 } }
I20260807 05:41:47.026381 11019 raft_consensus.cc:3037] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:41:47.033391 10832 raft_consensus.cc:1261] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 4 FOLLOWER]: Refusing update from remote peer c97de53a43de4f99bb1f74887936a162: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 05:41:47.036589 11009 consensus_queue.cc:1035] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1d4cb4dd704374aaee70e6341fbfdd" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:41:47.404573 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:41:47.425319 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:47.425894 10684 tablet_replica.cc:330] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd: stopping tablet replica
I20260807 05:41:47.426463 10684 raft_consensus.cc:2229] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:41:47.427079 10684 raft_consensus.cc:2258] T 2381e517a1b94797ab13438ddb52c5d5 P 9f1d4cb4dd704374aaee70e6341fbfdd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:47.429242 10684 tablet_replica.cc:330] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd: stopping tablet replica
I20260807 05:41:47.429666 10684 raft_consensus.cc:2229] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:41:47.430035 10684 raft_consensus.cc:2258] T 66b2f16ab002408ab2ff098a78b386ba P 9f1d4cb4dd704374aaee70e6341fbfdd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:47.587415 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:41:47.596510 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:41:47.613685 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:47.614233 10684 tablet_replica.cc:330] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162: stopping tablet replica
I20260807 05:41:47.614727 10684 raft_consensus.cc:2229] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 05:41:47.615302 10684 raft_consensus.cc:2258] T 2381e517a1b94797ab13438ddb52c5d5 P c97de53a43de4f99bb1f74887936a162 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:47.617075 10684 tablet_replica.cc:330] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162: stopping tablet replica
I20260807 05:41:47.617532 10684 raft_consensus.cc:2229] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 4 LEADER]: Raft consensus shutting down.
I20260807 05:41:47.618131 10684 raft_consensus.cc:2258] T 66b2f16ab002408ab2ff098a78b386ba P c97de53a43de4f99bb1f74887936a162 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:47.637130 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:41:47.644840 10684 master.cc:537] Master@127.10.111.62:39319 shutting down...
I20260807 05:41:47.805805 10781 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260807 05:41:47.829157 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:47.829654 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:47.829905 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P f81dba01fd354442a74735a542535164: stopping tablet replica
I20260807 05:41:47.846590 10684 master.cc:552] Master@127.10.111.62:39319 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7757 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260807 05:41:47.875025 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:38181
I20260807 05:41:47.876050 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:47.880820 11027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:47.882779 11028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:47.883116 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:47.883744 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:47.884301 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:47.884454 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:47.884563 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081307884555 us; error 0 us; skew 500 ppm
I20260807 05:41:47.885033 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:47.887221 10684 webserver.cc:460] Webserver started at http://127.10.111.62:39307/ using document root <none> and password file <none>
I20260807 05:41:47.887609 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:47.887774 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:47.887985 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:47.889025 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "1b88e65b4f0f44dba8330142bb4f24ba"
format_stamp: "Formatted at 2026-08-07 05:41:47 on dist-test-slave-qg60"
I20260807 05:41:47.893229 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:41:47.896171 11035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:47.896920 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:41:47.897207 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "1b88e65b4f0f44dba8330142bb4f24ba"
format_stamp: "Formatted at 2026-08-07 05:41:47 on dist-test-slave-qg60"
I20260807 05:41:47.897436 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:47.915608 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:47.916579 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:47.951033 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:38181
I20260807 05:41:47.951139 11089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:38181 every 8 connection(s)
I20260807 05:41:47.954686 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:47.963630 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba: Bootstrap starting.
I20260807 05:41:47.967442 11090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:47.971078 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba: No bootstrap required, opened a new log
I20260807 05:41:47.972934 11090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER }
I20260807 05:41:47.973297 11090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:47.973475 11090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b88e65b4f0f44dba8330142bb4f24ba, State: Initialized, Role: FOLLOWER
I20260807 05:41:47.973910 11090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER }
I20260807 05:41:47.974291 11090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:41:47.974486 11090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:41:47.974687 11090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:47.978417 11090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER }
I20260807 05:41:47.978864 11090 leader_election.cc:302] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b88e65b4f0f44dba8330142bb4f24ba; no voters:
I20260807 05:41:47.979887 11090 leader_election.cc:288] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:41:47.980319 11093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:47.981603 11093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 1 LEADER]: Becoming Leader. State: Replica: 1b88e65b4f0f44dba8330142bb4f24ba, State: Running, Role: LEADER
I20260807 05:41:47.982230 11093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER }
I20260807 05:41:47.982609 11090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:47.984771 11094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER } }
I20260807 05:41:47.985461 11094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:47.984913 11095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b88e65b4f0f44dba8330142bb4f24ba. Latest consensus state: current_term: 1 leader_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b88e65b4f0f44dba8330142bb4f24ba" member_type: VOTER } }
I20260807 05:41:47.986678 11095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:47.991041 11100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:47.996222 11100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:48.003917 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:48.006286 11100 catalog_manager.cc:1347] Generated new cluster ID: 96ef90ddb0a74144874b5bc2ee21a4df
I20260807 05:41:48.006572 11100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:48.031909 11100 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:48.033588 11100 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:48.063699 11100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba: Generated new TSK 0
I20260807 05:41:48.064352 11100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:48.070849 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:48.077093 11114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.078233 11115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.079814 11117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.080129 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:48.081027 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:48.081203 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:48.081314 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081308081306 us; error 0 us; skew 500 ppm
I20260807 05:41:48.081727 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:48.088336 10684 webserver.cc:460] Webserver started at http://127.10.111.1:33057/ using document root <none> and password file <none>
I20260807 05:41:48.088829 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:48.088999 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:48.089262 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:48.090346 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "7fd56752bb894423adba7979d4701385"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.094486 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:41:48.097661 11122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.098616 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:41:48.098884 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "7fd56752bb894423adba7979d4701385"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.099177 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:48.129052 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:48.130132 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:48.131528 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:48.133692 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:48.133898 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.134122 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:48.134275 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.171175 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:33065
I20260807 05:41:48.171288 11188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:33065 every 8 connection(s)
I20260807 05:41:48.175483 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:48.182412 11194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.183305 11195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.185968 11197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.186705 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:48.187481 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:48.187692 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:48.187880 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081308187866 us; error 0 us; skew 500 ppm
I20260807 05:41:48.188405 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:48.191571 10684 webserver.cc:460] Webserver started at http://127.10.111.2:36687/ using document root <none> and password file <none>
I20260807 05:41:48.191619 11190 heartbeater.cc:347] Connected to a master server at 127.10.111.62:38181
I20260807 05:41:48.192070 11190 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:48.192299 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:48.192553 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:48.192934 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:48.192883 11190 heartbeater.cc:511] Master 127.10.111.62:38181 requested a full tablet report, sending...
I20260807 05:41:48.194415 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "4b9b517124b9484994a71e45242cd1f4"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.195183 11054 ts_manager.cc:194] Registered new tserver with Master: 7fd56752bb894423adba7979d4701385 (127.10.111.1:33065)
I20260807 05:41:48.197569 11054 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46386
I20260807 05:41:48.199712 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 05:41:48.203128 11203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.203852 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:41:48.204169 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "4b9b517124b9484994a71e45242cd1f4"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.204473 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:48.225749 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:48.226832 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:48.228255 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:48.230490 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:48.230707 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.230932 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:48.231145 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.268081 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:44355
I20260807 05:41:48.268198 11271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:44355 every 8 connection(s)
I20260807 05:41:48.284013 11272 heartbeater.cc:347] Connected to a master server at 127.10.111.62:38181
I20260807 05:41:48.284405 11272 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:48.285171 11272 heartbeater.cc:511] Master 127.10.111.62:38181 requested a full tablet report, sending...
I20260807 05:41:48.287074 11054 ts_manager.cc:194] Registered new tserver with Master: 4b9b517124b9484994a71e45242cd1f4 (127.10.111.2:44355)
I20260807 05:41:48.287869 10684 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.016720924s
I20260807 05:41:48.288638 11054 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46400
I20260807 05:41:48.300565 11190 heartbeater.cc:503] Master 127.10.111.62:38181 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260807 05:41:48.313138 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:41:48.329099 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:48.344596 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:41:48.351364 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:41:48.366216 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:48.381668 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:41:48.388336 10684 master.cc:537] Master@127.10.111.62:38181 shutting down...
I20260807 05:41:48.403363 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:48.403869 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:48.404259 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1b88e65b4f0f44dba8330142bb4f24ba: stopping tablet replica
I20260807 05:41:48.412815 10684 master.cc:552] Master@127.10.111.62:38181 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (558 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260807 05:41:48.434068 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:33913
I20260807 05:41:48.435076 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:48.439685 11287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.441138 11288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.443190 10684 server_base.cc:1034] running on GCE node
W20260807 05:41:48.443621 11290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.444320 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:48.444514 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:48.444689 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081308444673 us; error 0 us; skew 500 ppm
I20260807 05:41:48.445237 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:48.447626 10684 webserver.cc:460] Webserver started at http://127.10.111.62:35869/ using document root <none> and password file <none>
I20260807 05:41:48.448101 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:48.448292 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:48.448546 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:48.449666 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "a39f2e9f1e8f4b0790e312dd25d8056f"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.454074 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:41:48.457321 11296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.458029 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:41:48.458284 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "a39f2e9f1e8f4b0790e312dd25d8056f"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.458575 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:48.477041 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:48.478049 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:48.512835 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:33913
I20260807 05:41:48.512955 11354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:33913 every 8 connection(s)
I20260807 05:41:48.516420 11355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:48.525282 11355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f: Bootstrap starting.
I20260807 05:41:48.528980 11355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:48.532771 11355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f: No bootstrap required, opened a new log
I20260807 05:41:48.534487 11355 raft_consensus.cc:348] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER }
I20260807 05:41:48.534847 11355 raft_consensus.cc:374] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:48.535247 11355 raft_consensus.cc:729] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39f2e9f1e8f4b0790e312dd25d8056f, State: Initialized, Role: FOLLOWER
I20260807 05:41:48.535846 11355 consensus_queue.cc:260] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER }
I20260807 05:41:48.536299 11355 raft_consensus.cc:388] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:41:48.536520 11355 raft_consensus.cc:482] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:41:48.536808 11355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:48.540503 11355 raft_consensus.cc:504] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER }
I20260807 05:41:48.541039 11355 leader_election.cc:302] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a39f2e9f1e8f4b0790e312dd25d8056f; no voters:
I20260807 05:41:48.542124 11355 leader_election.cc:288] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:41:48.542424 11359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:48.543694 11359 raft_consensus.cc:686] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 1 LEADER]: Becoming Leader. State: Replica: a39f2e9f1e8f4b0790e312dd25d8056f, State: Running, Role: LEADER
I20260807 05:41:48.544301 11359 consensus_queue.cc:237] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER }
I20260807 05:41:48.544907 11355 sys_catalog.cc:564] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:48.547019 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a39f2e9f1e8f4b0790e312dd25d8056f. Latest consensus state: current_term: 1 leader_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER } }
I20260807 05:41:48.546968 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39f2e9f1e8f4b0790e312dd25d8056f" member_type: VOTER } }
I20260807 05:41:48.547612 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:48.547686 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:48.553582 11364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:48.560827 11364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:48.561761 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:48.568662 11364 catalog_manager.cc:1347] Generated new cluster ID: 586a9f4dce5341d69b846505edf09ed2
I20260807 05:41:48.569016 11364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:48.589283 11364 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:48.591372 11364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:48.605386 11364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f: Generated new TSK 0
I20260807 05:41:48.605980 11364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:48.628031 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:48.634107 11380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.635157 11381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.636591 11383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.637465 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:48.638211 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:48.638382 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:48.638563 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081308638537 us; error 0 us; skew 500 ppm
I20260807 05:41:48.639046 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:48.641387 10684 webserver.cc:460] Webserver started at http://127.10.111.1:45123/ using document root <none> and password file <none>
I20260807 05:41:48.641836 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:48.642019 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:48.642274 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:48.643352 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "8290c87838214983bc909ff9bd1d1671"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.647799 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:41:48.650795 11388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.651494 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:41:48.651762 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "8290c87838214983bc909ff9bd1d1671"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.652015 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:48.663122 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:48.664073 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:48.665489 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:48.667568 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:48.667781 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.668016 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:48.668172 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.704895 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:38041
I20260807 05:41:48.705017 11453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:38041 every 8 connection(s)
I20260807 05:41:48.709514 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:48.716537 11459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.717168 11460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:48.720410 11463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:48.720495 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:48.721581 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:48.721836 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:48.722029 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081308722012 us; error 0 us; skew 500 ppm
I20260807 05:41:48.722764 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:48.724459 11454 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33913
I20260807 05:41:48.724833 11454 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:48.725461 10684 webserver.cc:460] Webserver started at http://127.10.111.2:40631/ using document root <none> and password file <none>
I20260807 05:41:48.725590 11454 heartbeater.cc:511] Master 127.10.111.62:33913 requested a full tablet report, sending...
I20260807 05:41:48.725970 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:48.726188 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:48.726461 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:48.727552 11315 ts_manager.cc:194] Registered new tserver with Master: 8290c87838214983bc909ff9bd1d1671 (127.10.111.1:38041)
I20260807 05:41:48.727802 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "4ea11832f8944548aeb4670999193a8b"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.730156 11315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58582
I20260807 05:41:48.733389 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260807 05:41:48.736474 11468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.737331 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:41:48.737599 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "4ea11832f8944548aeb4670999193a8b"
format_stamp: "Formatted at 2026-08-07 05:41:48 on dist-test-slave-qg60"
I20260807 05:41:48.737885 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:48.759757 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:48.760977 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:48.762449 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:48.764817 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:48.765050 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.765309 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:48.765501 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:48.805008 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:43829
I20260807 05:41:48.805094 11534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:43829 every 8 connection(s)
I20260807 05:41:48.819833 11535 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33913
I20260807 05:41:48.820169 11535 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:48.820811 11535 heartbeater.cc:511] Master 127.10.111.62:33913 requested a full tablet report, sending...
I20260807 05:41:48.822547 11315 ts_manager.cc:194] Registered new tserver with Master: 4ea11832f8944548aeb4670999193a8b (127.10.111.2:43829)
I20260807 05:41:48.822628 10684 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.014411259s
I20260807 05:41:48.824193 11315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58588
I20260807 05:41:48.833277 11454 heartbeater.cc:503] Master 127.10.111.62:33913 was elected leader, sending a full tablet report...
I20260807 05:41:48.846346 11315 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:41:48.847862 11315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260807 05:41:48.880632 11500 tablet_service.cc:1467] Processing CreateTablet for tablet e909b45bf362483b9b34bede6640c555 (DEFAULT_TABLE table=test-table [id=f766ab57ce094c0495cdac056be76b67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:48.881898 11500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e909b45bf362483b9b34bede6640c555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:48.882598 11418 tablet_service.cc:1467] Processing CreateTablet for tablet e909b45bf362483b9b34bede6640c555 (DEFAULT_TABLE table=test-table [id=f766ab57ce094c0495cdac056be76b67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:48.883733 11418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e909b45bf362483b9b34bede6640c555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:48.897256 11548 tablet_bootstrap.cc:492] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: Bootstrap starting.
I20260807 05:41:48.898540 11549 tablet_bootstrap.cc:492] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: Bootstrap starting.
I20260807 05:41:48.902365 11548 tablet_bootstrap.cc:654] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:48.902652 11549 tablet_bootstrap.cc:654] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:48.907533 11548 tablet_bootstrap.cc:492] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: No bootstrap required, opened a new log
I20260807 05:41:48.907660 11549 tablet_bootstrap.cc:492] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: No bootstrap required, opened a new log
I20260807 05:41:48.907949 11548 ts_tablet_manager.cc:1397] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260807 05:41:48.908054 11549 ts_tablet_manager.cc:1397] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 05:41:48.910010 11548 raft_consensus.cc:348] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:48.910092 11549 raft_consensus.cc:348] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:48.910672 11548 raft_consensus.cc:729] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea11832f8944548aeb4670999193a8b, State: Initialized, Role: FOLLOWER
I20260807 05:41:48.910750 11549 raft_consensus.cc:729] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Initialized, Role: FOLLOWER
I20260807 05:41:48.911252 11548 consensus_queue.cc:260] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:48.911271 11549 consensus_queue.cc:260] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:48.915475 11548 ts_tablet_manager.cc:1428] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 05:41:48.915267 11535 heartbeater.cc:503] Master 127.10.111.62:33913 was elected leader, sending a full tablet report...
I20260807 05:41:48.917454 11549 ts_tablet_manager.cc:1428] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20260807 05:41:48.947309 10684 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 05:41:48.947563 10684 raft_consensus.cc:412] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 0 FOLLOWER]: Emulating election...
I20260807 05:41:48.947813 10684 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:48.951749 10684 raft_consensus.cc:686] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 1 LEADER]: Becoming Leader. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Running, Role: LEADER
W20260807 05:41:48.952165 11454 heartbeater.cc:644] Failed to heartbeat to 127.10.111.62:33913 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260807 05:41:48.952461 10684 consensus_queue.cc:237] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:48.955476 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:48.977811 10684 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 05:41:49.081840 10684 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 05:41:49.286314 10684 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 05:41:49.406842 11510 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:49.412276 11555 consensus_queue.cc:1035] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:49.591424 10684 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 05:41:49.591696 10684 raft_consensus.cc:412] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 1 LEADER]: Emulating election...
I20260807 05:41:49.591933 10684 raft_consensus.cc:3032] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:49.592136 10684 raft_consensus.cc:729] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Running, Role: LEADER
I20260807 05:41:49.592587 10684 consensus_queue.cc:260] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:49.593272 10684 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:49.597254 10684 raft_consensus.cc:686] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 2 LEADER]: Becoming Leader. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Running, Role: LEADER
I20260807 05:41:49.597769 10684 consensus_queue.cc:237] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:49.599216 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:49.602809 10684 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 05:41:49.706704 10684 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 05:41:49.910852 10684 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 05:41:50.017865 11510 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:50.022442 11510 raft_consensus.cc:1261] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 2 FOLLOWER]: Refusing update from remote peer 8290c87838214983bc909ff9bd1d1671: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:50.023636 11560 consensus_queue.cc:1035] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:50.218814 10684 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 05:41:50.219241 10684 raft_consensus.cc:412] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 2 LEADER]: Emulating election...
I20260807 05:41:50.219578 10684 raft_consensus.cc:3032] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 05:41:50.219776 10684 raft_consensus.cc:729] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Running, Role: LEADER
I20260807 05:41:50.220492 10684 consensus_queue.cc:260] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:50.221750 10684 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:50.228011 10684 raft_consensus.cc:686] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 3 LEADER]: Becoming Leader. State: Replica: 8290c87838214983bc909ff9bd1d1671, State: Running, Role: LEADER
I20260807 05:41:50.228708 10684 consensus_queue.cc:237] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 } } peers { permanent_uuid: "8290c87838214983bc909ff9bd1d1671" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38041 } }
I20260807 05:41:50.230548 10684 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 05:41:50.234426 10684 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 05:41:50.338662 10684 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 05:41:50.542528 10684 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 05:41:50.660408 11510 raft_consensus.cc:3037] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:50.665921 11510 raft_consensus.cc:1261] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 3 FOLLOWER]: Refusing update from remote peer 8290c87838214983bc909ff9bd1d1671: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 05:41:50.667471 11560 consensus_queue.cc:1035] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea11832f8944548aeb4670999193a8b" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 05:41:50.852017 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:41:50.876011 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:50.876660 10684 tablet_replica.cc:330] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671: stopping tablet replica
I20260807 05:41:50.877293 10684 raft_consensus.cc:2229] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 3 LEADER]: Raft consensus shutting down.
I20260807 05:41:50.877954 10684 raft_consensus.cc:2258] T e909b45bf362483b9b34bede6640c555 P 8290c87838214983bc909ff9bd1d1671 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:50.897828 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:41:50.907176 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:41:50.928826 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:50.929522 10684 tablet_replica.cc:330] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b: stopping tablet replica
I20260807 05:41:50.930006 10684 raft_consensus.cc:2229] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 05:41:50.930426 10684 raft_consensus.cc:2258] T e909b45bf362483b9b34bede6640c555 P 4ea11832f8944548aeb4670999193a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:50.949695 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:41:50.959067 10684 master.cc:537] Master@127.10.111.62:33913 shutting down...
I20260807 05:41:50.976806 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:50.977366 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:50.977636 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P a39f2e9f1e8f4b0790e312dd25d8056f: stopping tablet replica
I20260807 05:41:50.995537 10684 master.cc:552] Master@127.10.111.62:33913 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2592 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10908 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260807 05:41:51.026654 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:34297
I20260807 05:41:51.027714 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:51.033839 11578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.034742 11579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.036197 11581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:51.036239 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:51.037267 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:51.037485 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:51.037645 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081311037629 us; error 0 us; skew 500 ppm
I20260807 05:41:51.038206 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:51.040863 10684 webserver.cc:460] Webserver started at http://127.10.111.62:46541/ using document root <none> and password file <none>
I20260807 05:41:51.041368 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:51.041572 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:51.041872 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:51.043052 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "7f3ffb8999e34a1b81db7fd55d188ef9"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.047662 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 05:41:51.051155 11586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.051864 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:41:51.052184 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "7f3ffb8999e34a1b81db7fd55d188ef9"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.052465 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:51.071067 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:51.072324 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:51.111068 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:34297
I20260807 05:41:51.111161 11641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:34297 every 8 connection(s)
I20260807 05:41:51.115072 11642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:51.125094 11642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9: Bootstrap starting.
I20260807 05:41:51.129403 11642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:51.134143 11642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9: No bootstrap required, opened a new log
I20260807 05:41:51.136164 11642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER }
I20260807 05:41:51.136556 11642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:51.136821 11642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f3ffb8999e34a1b81db7fd55d188ef9, State: Initialized, Role: FOLLOWER
I20260807 05:41:51.137373 11642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER }
I20260807 05:41:51.137811 11642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:41:51.138042 11642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:41:51.138319 11642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:51.142292 11642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER }
I20260807 05:41:51.142797 11642 leader_election.cc:302] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f3ffb8999e34a1b81db7fd55d188ef9; no voters:
I20260807 05:41:51.143925 11642 leader_election.cc:288] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:41:51.144353 11646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:51.145706 11646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 1 LEADER]: Becoming Leader. State: Replica: 7f3ffb8999e34a1b81db7fd55d188ef9, State: Running, Role: LEADER
I20260807 05:41:51.146351 11646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER }
I20260807 05:41:51.146890 11642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:51.149202 11647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER } }
I20260807 05:41:51.149711 11647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:51.149099 11648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f3ffb8999e34a1b81db7fd55d188ef9. Latest consensus state: current_term: 1 leader_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3ffb8999e34a1b81db7fd55d188ef9" member_type: VOTER } }
I20260807 05:41:51.150933 11648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:51.155587 11652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:51.161864 11652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:51.163105 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:51.170995 11652 catalog_manager.cc:1347] Generated new cluster ID: bad57572dab04aa89b882547db7880f0
I20260807 05:41:51.171294 11652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:51.196525 11652 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:51.198724 11652 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:51.215678 11652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9: Generated new TSK 0
I20260807 05:41:51.216354 11652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:51.229288 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:51.235314 11664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.236756 11665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.238494 11667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:51.239296 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:51.240031 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:51.240202 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:51.240311 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081311240303 us; error 0 us; skew 500 ppm
I20260807 05:41:51.240825 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:51.243147 10684 webserver.cc:460] Webserver started at http://127.10.111.1:36587/ using document root <none> and password file <none>
I20260807 05:41:51.243561 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:51.243705 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:51.243903 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:51.245079 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "bea1adf9450e408da11cf7f022a17e47"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.249835 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:41:51.253084 11672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.253836 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:41:51.254139 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "bea1adf9450e408da11cf7f022a17e47"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.254392 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:51.276499 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:51.277742 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:51.279150 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:51.281469 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:51.281641 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.281919 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:51.282109 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.321228 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:41131
I20260807 05:41:51.321317 11740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:41131 every 8 connection(s)
I20260807 05:41:51.326122 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:51.333999 11745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.335981 11746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.337880 11749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:51.338052 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:51.339202 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:51.339430 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:51.339603 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081311339584 us; error 0 us; skew 500 ppm
I20260807 05:41:51.340283 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:51.341984 11741 heartbeater.cc:347] Connected to a master server at 127.10.111.62:34297
I20260807 05:41:51.342342 11741 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:51.343062 10684 webserver.cc:460] Webserver started at http://127.10.111.2:37479/ using document root <none> and password file <none>
I20260807 05:41:51.343112 11741 heartbeater.cc:511] Master 127.10.111.62:34297 requested a full tablet report, sending...
I20260807 05:41:51.343601 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:51.343804 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:51.344121 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:51.345387 11604 ts_manager.cc:194] Registered new tserver with Master: bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131)
I20260807 05:41:51.345695 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "937c5811ffff441c9a3ea79af9e09109"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.348104 11604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53518
I20260807 05:41:51.351413 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 05:41:51.354852 11755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.355749 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:41:51.356063 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "937c5811ffff441c9a3ea79af9e09109"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.356377 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:51.376940 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:51.378153 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:51.379580 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:51.381726 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:51.381942 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.382182 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:51.382345 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.421895 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:35763
I20260807 05:41:51.422060 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:35763 every 8 connection(s)
I20260807 05:41:51.426522 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:51.433666 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.434525 11832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:51.439723 11838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:51.439834 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:51.441181 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:51.441457 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:51.441620 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081311441609 us; error 0 us; skew 500 ppm
I20260807 05:41:51.442193 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:51.442651 11827 heartbeater.cc:347] Connected to a master server at 127.10.111.62:34297
I20260807 05:41:51.443091 11827 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:51.443938 11827 heartbeater.cc:511] Master 127.10.111.62:34297 requested a full tablet report, sending...
I20260807 05:41:51.445060 10684 webserver.cc:460] Webserver started at http://127.10.111.3:37479/ using document root <none> and password file <none>
I20260807 05:41:51.445533 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:51.445716 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:51.445993 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:51.446167 11604 ts_manager.cc:194] Registered new tserver with Master: 937c5811ffff441c9a3ea79af9e09109 (127.10.111.2:35763)
I20260807 05:41:51.447273 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-2-root/instance:
uuid: "c723e2a7988a4134b952d108848d6891"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.448232 11604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53526
I20260807 05:41:51.451956 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 05:41:51.455655 11843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.456801 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:41:51.457104 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-2-root
uuid: "c723e2a7988a4134b952d108848d6891"
format_stamp: "Formatted at 2026-08-07 05:41:51 on dist-test-slave-qg60"
I20260807 05:41:51.457438 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786081299864053-10684-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:51.467409 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:51.468570 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:51.470013 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:51.472122 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:51.472332 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:51.472589 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:51.472771 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 05:41:51.542902 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.3:33527
I20260807 05:41:51.543077 11914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.3:33527 every 8 connection(s)
I20260807 05:41:51.560956 11915 heartbeater.cc:347] Connected to a master server at 127.10.111.62:34297
I20260807 05:41:51.561391 11915 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:51.562139 11915 heartbeater.cc:511] Master 127.10.111.62:34297 requested a full tablet report, sending...
I20260807 05:41:51.563966 11604 ts_manager.cc:194] Registered new tserver with Master: c723e2a7988a4134b952d108848d6891 (127.10.111.3:33527)
I20260807 05:41:51.564107 10684 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016096475s
I20260807 05:41:51.565587 11604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53540
I20260807 05:41:51.590530 11604 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260807 05:41:51.592008 11604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:41:51.641426 11786 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfabfb9243c4a47937c4a66bdf4029c (DEFAULT_TABLE table=test-table [id=5d3c2d8b22a745a4b555fa11c6a2b784]), partition=
I20260807 05:41:51.642402 11786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfabfb9243c4a47937c4a66bdf4029c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:51.643311 11874 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfabfb9243c4a47937c4a66bdf4029c (DEFAULT_TABLE table=test-table [id=5d3c2d8b22a745a4b555fa11c6a2b784]), partition=
I20260807 05:41:51.644501 11874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfabfb9243c4a47937c4a66bdf4029c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:51.646231 11702 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfabfb9243c4a47937c4a66bdf4029c (DEFAULT_TABLE table=test-table [id=5d3c2d8b22a745a4b555fa11c6a2b784]), partition=
I20260807 05:41:51.647320 11702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfabfb9243c4a47937c4a66bdf4029c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:51.661480 11932 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: Bootstrap starting.
I20260807 05:41:51.665788 11931 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: Bootstrap starting.
I20260807 05:41:51.666484 11932 tablet_bootstrap.cc:654] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:51.666584 11930 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: Bootstrap starting.
I20260807 05:41:51.673197 11931 tablet_bootstrap.cc:654] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:51.673954 11930 tablet_bootstrap.cc:654] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:51.674257 11932 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: No bootstrap required, opened a new log
I20260807 05:41:51.674755 11932 ts_tablet_manager.cc:1397] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260807 05:41:51.679339 11931 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: No bootstrap required, opened a new log
I20260807 05:41:51.679747 11931 ts_tablet_manager.cc:1397] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260807 05:41:51.678987 11932 raft_consensus.cc:348] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.681553 11932 raft_consensus.cc:374] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:51.681890 11932 raft_consensus.cc:729] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea1adf9450e408da11cf7f022a17e47, State: Initialized, Role: FOLLOWER
I20260807 05:41:51.682977 11930 tablet_bootstrap.cc:492] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: No bootstrap required, opened a new log
I20260807 05:41:51.682667 11932 consensus_queue.cc:260] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.682641 11931 raft_consensus.cc:348] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.683454 11930 ts_tablet_manager.cc:1397] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20260807 05:41:51.683660 11931 raft_consensus.cc:374] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:51.684170 11931 raft_consensus.cc:729] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937c5811ffff441c9a3ea79af9e09109, State: Initialized, Role: FOLLOWER
I20260807 05:41:51.685343 11931 consensus_queue.cc:260] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.686137 11741 heartbeater.cc:503] Master 127.10.111.62:34297 was elected leader, sending a full tablet report...
I20260807 05:41:51.686708 11932 ts_tablet_manager.cc:1428] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260807 05:41:51.686316 11930 raft_consensus.cc:348] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.687191 11930 raft_consensus.cc:374] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:51.687480 11930 raft_consensus.cc:729] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c723e2a7988a4134b952d108848d6891, State: Initialized, Role: FOLLOWER
I20260807 05:41:51.688221 11930 consensus_queue.cc:260] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.691866 11915 heartbeater.cc:503] Master 127.10.111.62:34297 was elected leader, sending a full tablet report...
I20260807 05:41:51.692131 11930 ts_tablet_manager.cc:1428] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 05:41:51.694170 11827 heartbeater.cc:503] Master 127.10.111.62:34297 was elected leader, sending a full tablet report...
I20260807 05:41:51.694741 11931 ts_tablet_manager.cc:1428] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260807 05:41:51.775313 11937 raft_consensus.cc:482] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:51.775889 11937 raft_consensus.cc:504] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.778412 11937 leader_election.cc:288] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c723e2a7988a4134b952d108848d6891 (127.10.111.3:33527), 937c5811ffff441c9a3ea79af9e09109 (127.10.111.2:35763)
I20260807 05:41:51.789711 11884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "bea1adf9450e408da11cf7f022a17e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c723e2a7988a4134b952d108848d6891" is_pre_election: true
I20260807 05:41:51.790951 11884 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea1adf9450e408da11cf7f022a17e47 in term 0.
I20260807 05:41:51.792407 11676 leader_election.cc:302] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bea1adf9450e408da11cf7f022a17e47, c723e2a7988a4134b952d108848d6891; no voters:
I20260807 05:41:51.793377 11937 raft_consensus.cc:2781] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:51.793715 11937 raft_consensus.cc:482] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:51.793684 11798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "bea1adf9450e408da11cf7f022a17e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937c5811ffff441c9a3ea79af9e09109" is_pre_election: true
I20260807 05:41:51.794209 11937 raft_consensus.cc:3037] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:51.794502 11798 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea1adf9450e408da11cf7f022a17e47 in term 0.
I20260807 05:41:51.799358 11937 raft_consensus.cc:504] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.801047 11937 leader_election.cc:288] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [CANDIDATE]: Term 1 election: Requested vote from peers c723e2a7988a4134b952d108848d6891 (127.10.111.3:33527), 937c5811ffff441c9a3ea79af9e09109 (127.10.111.2:35763)
I20260807 05:41:51.802076 11884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "bea1adf9450e408da11cf7f022a17e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c723e2a7988a4134b952d108848d6891"
I20260807 05:41:51.802140 11798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "bea1adf9450e408da11cf7f022a17e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937c5811ffff441c9a3ea79af9e09109"
I20260807 05:41:51.802636 11884 raft_consensus.cc:3037] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:51.802711 11798 raft_consensus.cc:3037] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:51.807557 11798 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea1adf9450e408da11cf7f022a17e47 in term 1.
I20260807 05:41:51.807551 11884 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea1adf9450e408da11cf7f022a17e47 in term 1.
I20260807 05:41:51.808712 11673 leader_election.cc:302] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 937c5811ffff441c9a3ea79af9e09109, bea1adf9450e408da11cf7f022a17e47; no voters:
I20260807 05:41:51.809448 11937 raft_consensus.cc:2781] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:51.810252 11937 raft_consensus.cc:686] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 1 LEADER]: Becoming Leader. State: Replica: bea1adf9450e408da11cf7f022a17e47, State: Running, Role: LEADER
I20260807 05:41:51.811030 11937 consensus_queue.cc:237] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:51.818353 11602 catalog_manager.cc:5495] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 reported cstate change: term changed from 0 to 1, leader changed from <none> to bea1adf9450e408da11cf7f022a17e47 (127.10.111.1). New cstate: current_term: 1 leader_uuid: "bea1adf9450e408da11cf7f022a17e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } health_report { overall_health: HEALTHY } } }
I20260807 05:41:52.270433 11937 consensus_queue.cc:1035] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:52.281908 11947 consensus_queue.cc:1035] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:52.885456 10684 tablet_replica.cc:330] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47: stopping tablet replica
I20260807 05:41:52.886122 10684 raft_consensus.cc:2229] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:52.886914 10684 raft_consensus.cc:2258] T 3dfabfb9243c4a47937c4a66bdf4029c P bea1adf9450e408da11cf7f022a17e47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:54.269798 11952 raft_consensus.cc:482] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bea1adf9450e408da11cf7f022a17e47)
I20260807 05:41:54.270320 11952 raft_consensus.cc:504] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:54.272257 11952 leader_election.cc:288] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c723e2a7988a4134b952d108848d6891 (127.10.111.3:33527), bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131)
I20260807 05:41:54.292819 11884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "937c5811ffff441c9a3ea79af9e09109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c723e2a7988a4134b952d108848d6891" is_pre_election: true
I20260807 05:41:54.292822 11713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "937c5811ffff441c9a3ea79af9e09109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bea1adf9450e408da11cf7f022a17e47" is_pre_election: true
W20260807 05:41:54.294500 11759 leader_election.cc:341] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131): Illegal state: cannot vote while shut down
I20260807 05:41:54.294915 11759 leader_election.cc:302] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 937c5811ffff441c9a3ea79af9e09109; no voters: bea1adf9450e408da11cf7f022a17e47, c723e2a7988a4134b952d108848d6891
I20260807 05:41:54.295506 11952 raft_consensus.cc:2726] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:41:54.341800 11957 raft_consensus.cc:482] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bea1adf9450e408da11cf7f022a17e47)
I20260807 05:41:54.342321 11957 raft_consensus.cc:504] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:54.344228 11957 leader_election.cc:288] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 937c5811ffff441c9a3ea79af9e09109 (127.10.111.2:35763), bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131)
I20260807 05:41:54.362510 11713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "c723e2a7988a4134b952d108848d6891" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bea1adf9450e408da11cf7f022a17e47" is_pre_election: true
W20260807 05:41:54.363758 11847 leader_election.cc:341] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131): Illegal state: cannot vote while shut down
I20260807 05:41:54.363812 11798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "c723e2a7988a4134b952d108848d6891" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "937c5811ffff441c9a3ea79af9e09109" is_pre_election: true
I20260807 05:41:54.364524 11798 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c723e2a7988a4134b952d108848d6891 in term 1.
I20260807 05:41:54.365741 11844 leader_election.cc:302] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 937c5811ffff441c9a3ea79af9e09109, c723e2a7988a4134b952d108848d6891; no voters: bea1adf9450e408da11cf7f022a17e47
I20260807 05:41:54.366576 11957 raft_consensus.cc:2781] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:41:54.366829 11957 raft_consensus.cc:482] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bea1adf9450e408da11cf7f022a17e47)
I20260807 05:41:54.367122 11957 raft_consensus.cc:3037] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:54.371685 11957 raft_consensus.cc:504] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:54.373148 11957 leader_election.cc:288] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 election: Requested vote from peers 937c5811ffff441c9a3ea79af9e09109 (127.10.111.2:35763), bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131)
I20260807 05:41:54.373986 11798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "c723e2a7988a4134b952d108848d6891" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "937c5811ffff441c9a3ea79af9e09109"
I20260807 05:41:54.374195 11713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfabfb9243c4a47937c4a66bdf4029c" candidate_uuid: "c723e2a7988a4134b952d108848d6891" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bea1adf9450e408da11cf7f022a17e47"
I20260807 05:41:54.374444 11798 raft_consensus.cc:3037] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 1 FOLLOWER]: Advancing to term 2
W20260807 05:41:54.375295 11847 leader_election.cc:341] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131): Illegal state: cannot vote while shut down
I20260807 05:41:54.378800 11798 raft_consensus.cc:2454] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c723e2a7988a4134b952d108848d6891 in term 2.
I20260807 05:41:54.379676 11844 leader_election.cc:302] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 937c5811ffff441c9a3ea79af9e09109, c723e2a7988a4134b952d108848d6891; no voters: bea1adf9450e408da11cf7f022a17e47
I20260807 05:41:54.380409 11957 raft_consensus.cc:2781] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:41:54.381609 11957 raft_consensus.cc:686] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 2 LEADER]: Becoming Leader. State: Replica: c723e2a7988a4134b952d108848d6891, State: Running, Role: LEADER
I20260807 05:41:54.382341 11957 consensus_queue.cc:237] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c723e2a7988a4134b952d108848d6891" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 33527 } } peers { permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 } } peers { permanent_uuid: "bea1adf9450e408da11cf7f022a17e47" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 41131 } }
I20260807 05:41:54.810398 11847 consensus_queue.cc:579] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [LEADER]: The tablet replica hosted on peer bea1adf9450e408da11cf7f022a17e47 has failed
W20260807 05:41:54.810999 11847 consensus_peers.cc:487] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 -> Peer bea1adf9450e408da11cf7f022a17e47 (127.10.111.1:41131): Couldn't send request to peer bea1adf9450e408da11cf7f022a17e47. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:41:54.860020 11798 raft_consensus.cc:1261] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 2 FOLLOWER]: Refusing update from remote peer c723e2a7988a4134b952d108848d6891: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:54.861416 11961 consensus_queue.cc:1035] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937c5811ffff441c9a3ea79af9e09109" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 35763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:54.922851 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:41:54.943038 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:54.960855 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:41:54.968598 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:41:54.985153 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:54.985713 10684 tablet_replica.cc:330] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109: stopping tablet replica
I20260807 05:41:54.986213 10684 raft_consensus.cc:2229] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 05:41:54.986615 10684 raft_consensus.cc:2258] T 3dfabfb9243c4a47937c4a66bdf4029c P 937c5811ffff441c9a3ea79af9e09109 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:55.005650 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:41:55.013888 10684 tablet_server.cc:178] TabletServer@127.10.111.3:0 shutting down...
I20260807 05:41:55.030102 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:55.030675 10684 tablet_replica.cc:330] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891: stopping tablet replica
I20260807 05:41:55.031142 10684 raft_consensus.cc:2229] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:41:55.031850 10684 raft_consensus.cc:2258] T 3dfabfb9243c4a47937c4a66bdf4029c P c723e2a7988a4134b952d108848d6891 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:55.050643 10684 tablet_server.cc:194] TabletServer@127.10.111.3:0 shutdown complete.
I20260807 05:41:55.059294 10684 master.cc:537] Master@127.10.111.62:34297 shutting down...
I20260807 05:41:55.073544 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:55.074090 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:55.074410 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7f3ffb8999e34a1b81db7fd55d188ef9: stopping tablet replica
I20260807 05:41:55.091660 10684 master.cc:552] Master@127.10.111.62:34297 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (4089 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260807 05:41:55.115139 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:41853
I20260807 05:41:55.116165 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:55.121232 11971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.122341 11972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.125388 11975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.125589 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:55.126402 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.126636 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.126852 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315126831 us; error 0 us; skew 500 ppm
I20260807 05:41:55.127457 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.135051 10684 webserver.cc:460] Webserver started at http://127.10.111.62:40683/ using document root <none> and password file <none>
I20260807 05:41:55.135597 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.135818 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:55.136155 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:55.137415 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "af6d993e93484917af5a96423ab6d142"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.142172 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:41:55.145370 11980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.146149 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:41:55.146428 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "af6d993e93484917af5a96423ab6d142"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.146725 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.160022 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.161116 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.197860 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:41853
I20260807 05:41:55.197933 12036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:41853 every 8 connection(s)
I20260807 05:41:55.201557 12037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:55.210978 12037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142: Bootstrap starting.
I20260807 05:41:55.215197 12037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:55.218854 12037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142: No bootstrap required, opened a new log
I20260807 05:41:55.220559 12037 raft_consensus.cc:348] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER }
I20260807 05:41:55.220943 12037 raft_consensus.cc:374] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:55.221204 12037 raft_consensus.cc:729] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6d993e93484917af5a96423ab6d142, State: Initialized, Role: FOLLOWER
I20260807 05:41:55.221764 12037 consensus_queue.cc:260] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER }
I20260807 05:41:55.222185 12037 raft_consensus.cc:388] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:41:55.222386 12037 raft_consensus.cc:482] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:41:55.222669 12037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:55.226868 12037 raft_consensus.cc:504] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER }
I20260807 05:41:55.227466 12037 leader_election.cc:302] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af6d993e93484917af5a96423ab6d142; no voters:
I20260807 05:41:55.228597 12037 leader_election.cc:288] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:41:55.229043 12040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:55.230445 12040 raft_consensus.cc:686] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 1 LEADER]: Becoming Leader. State: Replica: af6d993e93484917af5a96423ab6d142, State: Running, Role: LEADER
I20260807 05:41:55.231137 12040 consensus_queue.cc:237] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER }
I20260807 05:41:55.231465 12037 sys_catalog.cc:564] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:55.234195 12042 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6d993e93484917af5a96423ab6d142. Latest consensus state: current_term: 1 leader_uuid: "af6d993e93484917af5a96423ab6d142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER } }
I20260807 05:41:55.234943 12042 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:55.234143 12041 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af6d993e93484917af5a96423ab6d142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6d993e93484917af5a96423ab6d142" member_type: VOTER } }
I20260807 05:41:55.235455 12041 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:55.238973 12045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:55.246948 12045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:55.247931 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:55.255662 12045 catalog_manager.cc:1347] Generated new cluster ID: b3144bff4c49479b92645661e8ea1a81
I20260807 05:41:55.255976 12045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:55.299407 12045 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:55.301090 12045 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:55.312685 12045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142: Generated new TSK 0
I20260807 05:41:55.313467 12045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:55.378794 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:55.384930 12058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.386415 12059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.389443 12061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.390237 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:55.391005 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.391187 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.391310 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315391301 us; error 0 us; skew 500 ppm
I20260807 05:41:55.391785 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.394296 10684 webserver.cc:460] Webserver started at http://127.10.111.1:36839/ using document root <none> and password file <none>
I20260807 05:41:55.394770 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.394933 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:55.395154 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:55.396234 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "536e2a40f97b4e03bd3fd1c941c305b5"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.401036 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:41:55.404556 12066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.405349 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:41:55.405655 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "536e2a40f97b4e03bd3fd1c941c305b5"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.405926 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.425109 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.426260 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.427734 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:55.430186 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:55.430401 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.430701 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:55.430876 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.473438 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:36867
I20260807 05:41:55.473544 12134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:36867 every 8 connection(s)
I20260807 05:41:55.489441 12135 heartbeater.cc:347] Connected to a master server at 127.10.111.62:41853
I20260807 05:41:55.489769 12135 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:55.490375 12135 heartbeater.cc:511] Master 127.10.111.62:41853 requested a full tablet report, sending...
I20260807 05:41:55.492123 12000 ts_manager.cc:194] Registered new tserver with Master: 536e2a40f97b4e03bd3fd1c941c305b5 (127.10.111.1:36867)
I20260807 05:41:55.492690 10684 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016191049s
I20260807 05:41:55.494001 12000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53570
I20260807 05:41:55.515143 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:41:55.530961 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:55.548804 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:41:55.556099 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:41:55.558553 10684 tablet_server.cc:93] deduplicated master addresses: 127.10.111.62:41853
W20260807 05:41:55.561599 12147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.562448 12148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.563622 12150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.564513 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:55.565445 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.565621 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.565773 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315565760 us; error 0 us; skew 500 ppm
I20260807 05:41:55.566223 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.568507 10684 webserver.cc:460] Webserver started at http://127.10.111.1:36839/ using document root <none> and password file <none>
I20260807 05:41:55.569000 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.569149 10684 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 05:41:55.572618 10684 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:41:55.575724 12155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.576507 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:41:55.576912 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "536e2a40f97b4e03bd3fd1c941c305b5"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.577188 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.589468 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.590390 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.591759 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:55.594002 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:55.594178 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.594458 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:55.594635 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.633859 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:36867
I20260807 05:41:55.634061 12223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:36867 every 8 connection(s)
I20260807 05:41:55.640763 10684 tablet_server.cc:178] TabletServer@127.10.111.1:36867 shutting down...
I20260807 05:41:55.656776 12224 heartbeater.cc:347] Connected to a master server at 127.10.111.62:41853
I20260807 05:41:55.657195 12224 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:55.659101 12224 heartbeater.cc:511] Master 127.10.111.62:41853 requested a full tablet report, sending...
I20260807 05:41:55.662310 12000 ts_manager.cc:194] Re-registered known tserver with Master: 536e2a40f97b4e03bd3fd1c941c305b5 (127.10.111.1:36867)
I20260807 05:41:55.664634 12000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53596
I20260807 05:41:55.667448 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:41:55.683061 10684 tablet_server.cc:194] TabletServer@127.10.111.1:36867 shutdown complete.
I20260807 05:41:55.690193 10684 master.cc:537] Master@127.10.111.62:41853 shutting down...
I20260807 05:41:55.703898 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:41:55.704391 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:41:55.704882 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P af6d993e93484917af5a96423ab6d142: stopping tablet replica
I20260807 05:41:55.717911 10684 master.cc:552] Master@127.10.111.62:41853 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (625 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260807 05:41:55.739790 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:40189,127.10.111.61:34761,127.10.111.60:42959
I20260807 05:41:55.741026 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:55.745925 12231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.747051 12232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.748898 10684 server_base.cc:1034] running on GCE node
W20260807 05:41:55.748934 12234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.749825 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.750025 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.750192 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315750178 us; error 0 us; skew 500 ppm
I20260807 05:41:55.750756 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.753214 10684 webserver.cc:460] Webserver started at http://127.10.111.62:35697/ using document root <none> and password file <none>
I20260807 05:41:55.753674 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.753867 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:55.754137 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:55.755178 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "783781150ce54c45b48341c0f1386ebb"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.759465 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 05:41:55.762666 12239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:55.763548 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:41:55.763785 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "783781150ce54c45b48341c0f1386ebb"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.764039 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.775720 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.776656 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.812860 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:40189
I20260807 05:41:55.812919 12298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:40189 every 8 connection(s)
I20260807 05:41:55.816686 12299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:55.817113 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:55.822252 12302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.823261 12303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.824790 12299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } has no permanent_uuid. Determining permanent_uuid...
W20260807 05:41:55.829408 12306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.830758 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:55.831698 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.831948 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.832149 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315832130 us; error 0 us; skew 500 ppm
I20260807 05:41:55.832887 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.835577 10684 webserver.cc:460] Webserver started at http://127.10.111.61:35721/ using document root <none> and password file <none>
I20260807 05:41:55.836174 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.836409 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:55.836720 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:55.838083 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-1-root/instance:
uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.843109 12299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:55.843715 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
W20260807 05:41:55.845441 12242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.111.61:34761: connect: Connection refused (error 111)
I20260807 05:41:55.848495 12316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260807 05:41:55.849231 12299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.61:34761: Network error: Client connection negotiation failed: client connection to 127.10.111.61:34761: connect: Connection refused (error 111)
I20260807 05:41:55.849690 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:41:55.850005 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-1-root
uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.850256 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.879012 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.879973 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.887977 12299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } attempt: 1
W20260807 05:41:55.893162 12299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.61:34761: Network error: Client connection negotiation failed: client connection to 127.10.111.61:34761: connect: Connection refused (error 111)
I20260807 05:41:55.917528 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.61:34761
I20260807 05:41:55.917640 12371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.61:34761 every 8 connection(s)
I20260807 05:41:55.921336 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:41:55.921329 12372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 05:41:55.927207 12374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:55.929072 12376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.929751 12372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } has no permanent_uuid. Determining permanent_uuid...
W20260807 05:41:55.932272 12378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:55.933706 12299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } attempt: 2
I20260807 05:41:55.934074 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:55.934955 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:55.935237 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:55.935459 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081315935441 us; error 0 us; skew 500 ppm
I20260807 05:41:55.936189 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:55.940524 10684 webserver.cc:460] Webserver started at http://127.10.111.60:42695/ using document root <none> and password file <none>
I20260807 05:41:55.941143 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:55.941403 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:55.941735 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:55.943032 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root/instance:
uuid: "1e2b5735fd7d470e866112a36570d269"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.949326 12372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:55.949844 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 05:41:55.951835 12299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:55.955858 12385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260807 05:41:55.956108 12299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.60:42959: Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:41:55.957010 10684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:41:55.957286 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
uuid: "1e2b5735fd7d470e866112a36570d269"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:41:55.957571 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:55.963639 12372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
W20260807 05:41:55.968647 12372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.60:42959: Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:41:55.973152 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:55.974108 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:55.991667 12299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } attempt: 1
W20260807 05:41:55.996676 12299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.60:42959: Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:41:56.002192 12372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } attempt: 1
W20260807 05:41:56.007572 12372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.111.60:42959: Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:41:56.013233 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.60:42959
I20260807 05:41:56.013566 12445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.60:42959 every 8 connection(s)
I20260807 05:41:56.016361 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:41:56.017170 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:56.022909 12446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:56.035374 12446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:56.046236 12446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:41:56.046135 12372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } attempt: 2
I20260807 05:41:56.072268 12299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } attempt: 2
I20260807 05:41:56.072968 12446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Bootstrap starting.
I20260807 05:41:56.073205 12372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: Bootstrap starting.
I20260807 05:41:56.078125 12372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:56.078531 12446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:56.083159 12372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: No bootstrap required, opened a new log
I20260807 05:41:56.084378 12446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: No bootstrap required, opened a new log
I20260807 05:41:56.085664 12372 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.086337 12372 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:56.086602 12372 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd137409d1b9434e8fce87acd2f0f8e6, State: Initialized, Role: FOLLOWER
I20260807 05:41:56.086849 12446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.087311 12372 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.087674 12446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:56.088070 12446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2b5735fd7d470e866112a36570d269, State: Initialized, Role: FOLLOWER
I20260807 05:41:56.088816 12446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.091105 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.092253 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.092595 12446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:56.093766 12453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.094412 12299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: Bootstrap starting.
I20260807 05:41:56.094681 12453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.095346 12372 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:41:56.101765 12299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:56.108054 12299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: No bootstrap required, opened a new log
I20260807 05:41:56.111485 12299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.112485 12299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:56.112875 12299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783781150ce54c45b48341c0f1386ebb, State: Initialized, Role: FOLLOWER
W20260807 05:41:56.113397 12474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:41:56.113758 12474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:41:56.113725 12299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.116427 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.117255 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.118098 12299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:41:56.119963 12479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:41:56.120237 12479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:41:56.127063 12478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:56.127509 12478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.129391 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260807 05:41:56.129665 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260807 05:41:56.129321 12478 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:41:56.130471 12416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269" is_pre_election: true
I20260807 05:41:56.130467 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" is_pre_election: true
I20260807 05:41:56.131429 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:41:56.131362 12416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 0.
I20260807 05:41:56.131856 12347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 0.
I20260807 05:41:56.132472 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2b5735fd7d470e866112a36570d269, 783781150ce54c45b48341c0f1386ebb; no voters:
I20260807 05:41:56.133352 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:56.133733 12478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:56.134061 12478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 0 FOLLOWER]: Advancing to term 1
W20260807 05:41:56.134344 12492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:41:56.134625 12492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 05:41:56.140548 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:56.141733 12494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.142225 12478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.143914 12478 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 1 election: Requested vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:41:56.144951 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
I20260807 05:41:56.145371 12416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269"
I20260807 05:41:56.145521 12347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:56.145911 12416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:56.148761 10684 server_base.cc:1034] running on GCE node
W20260807 05:41:56.152264 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.152633 12347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 1.
I20260807 05:41:56.152699 12416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 1.
I20260807 05:41:56.153829 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 05:41:56.153776 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6; no voters:
W20260807 05:41:56.154191 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:56.154398 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081316154369 us; error 0 us; skew 500 ppm
I20260807 05:41:56.154567 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:56.155058 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:56.156165 12478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 LEADER]: Becoming Leader. State: Replica: 783781150ce54c45b48341c0f1386ebb, State: Running, Role: LEADER
I20260807 05:41:56.157017 12478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:56.158506 10684 webserver.cc:460] Webserver started at http://127.10.111.1:33925/ using document root <none> and password file <none>
I20260807 05:41:56.159080 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:56.159282 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:56.159605 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:56.160360 12498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.161041 12498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:56.161229 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "e8f9fe91e3164e39bb3d65835859ba06"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.162683 12502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:56.166656 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 05:41:56.167877 12502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:41:56.170478 12504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.171325 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:41:56.171613 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "e8f9fe91e3164e39bb3d65835859ba06"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.171901 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:56.176807 12416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Refusing update from remote peer 783781150ce54c45b48341c0f1386ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:41:56.176807 12347 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Refusing update from remote peer 783781150ce54c45b48341c0f1386ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:41:56.178824 12498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:56.179674 12478 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:41:56.190061 12453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.190836 12453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.195139 12505 mvcc.cc:205] Tried to move back new op lower bound from 7315789071046545408 to 7315789070986190848. Current Snapshot: MvccSnapshot[applied={T|T < 7315789071046545408}]
I20260807 05:41:56.195330 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.196173 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.205045 12453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.205832 12453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.203522 12502 catalog_manager.cc:1347] Generated new cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:41:56.208204 12502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:56.211905 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.212646 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:56.213336 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.214026 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:56.214973 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:56.215677 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:56.224522 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:56.225656 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:56.227193 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:56.229895 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:56.230084 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.230362 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:56.230525 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.246177 12502 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:41:56.248598 12502 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:56.281870 12502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: Generated new TSK 0
I20260807 05:41:56.282918 12502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:56.314626 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:46737
I20260807 05:41:56.314756 12580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:46737 every 8 connection(s)
I20260807 05:41:56.323144 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:41:56.336678 12589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:41:56.344556 12590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.347319 12581 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
I20260807 05:41:56.347800 12581 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.348843 12581 heartbeater.cc:511] Master 127.10.111.60:42959 requested a full tablet report, sending...
I20260807 05:41:56.352399 12405 ts_manager.cc:194] Registered new tserver with Master: e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:41:56.353124 12582 heartbeater.cc:347] Connected to a master server at 127.10.111.61:34761
I20260807 05:41:56.353644 12582 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.354681 12582 heartbeater.cc:511] Master 127.10.111.61:34761 requested a full tablet report, sending...
I20260807 05:41:56.357826 12336 ts_manager.cc:194] Registered new tserver with Master: e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
W20260807 05:41:56.360886 12592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.363134 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:56.364234 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 05:41:56.364234 12583 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40189
W20260807 05:41:56.364677 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:56.364974 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081316364956 us; error 0 us; skew 500 ppm
I20260807 05:41:56.364991 12583 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.365857 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:56.366276 12583 heartbeater.cc:511] Master 127.10.111.62:40189 requested a full tablet report, sending...
I20260807 05:41:56.369684 10684 webserver.cc:460] Webserver started at http://127.10.111.2:35765/ using document root <none> and password file <none>
I20260807 05:41:56.370369 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:56.370657 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:56.371006 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:56.371260 12258 ts_manager.cc:194] Registered new tserver with Master: e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:41:56.372612 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "56379265ad0445b3af80f95324dc276a"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.374190 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34096
I20260807 05:41:56.379309 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 05:41:56.383715 12597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.384512 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:41:56.384830 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "56379265ad0445b3af80f95324dc276a"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.385085 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:56.398841 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:56.400938 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:56.405014 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:56.409443 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:56.409687 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.409941 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:56.410120 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.478842 12583 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:41:56.545279 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:39769
I20260807 05:41:56.545423 12663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:39769 every 8 connection(s)
I20260807 05:41:56.572340 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:41:56.593425 12664 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
W20260807 05:41:56.593473 12672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.594055 12664 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.595534 12664 heartbeater.cc:511] Master 127.10.111.60:42959 requested a full tablet report, sending...
W20260807 05:41:56.597771 12673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.599874 12405 ts_manager.cc:194] Registered new tserver with Master: 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:56.604511 12665 heartbeater.cc:347] Connected to a master server at 127.10.111.61:34761
I20260807 05:41:56.605171 12665 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.606781 12665 heartbeater.cc:511] Master 127.10.111.61:34761 requested a full tablet report, sending...
I20260807 05:41:56.610323 10684 server_base.cc:1034] running on GCE node
I20260807 05:41:56.610795 12336 ts_manager.cc:194] Registered new tserver with Master: 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:56.612692 12668 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40189
I20260807 05:41:56.613165 12668 heartbeater.cc:464] Registering TS with master...
W20260807 05:41:56.613512 12677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:41:56.613977 12668 heartbeater.cc:511] Master 127.10.111.62:40189 requested a full tablet report, sending...
I20260807 05:41:56.614761 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:41:56.615082 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:41:56.615298 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081316615283 us; error 0 us; skew 500 ppm
I20260807 05:41:56.616135 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:41:56.616628 12258 ts_manager.cc:194] Registered new tserver with Master: 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:56.619323 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34100
I20260807 05:41:56.619562 10684 webserver.cc:460] Webserver started at http://127.10.111.3:43647/ using document root <none> and password file <none>
I20260807 05:41:56.620260 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:41:56.620471 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:41:56.620858 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:41:56.622001 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-2-root/instance:
uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.627249 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 05:41:56.631079 12681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.631866 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:41:56.632194 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-2-root
uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
format_stamp: "Formatted at 2026-08-07 05:41:56 on dist-test-slave-qg60"
I20260807 05:41:56.632515 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:41:56.667785 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:41:56.669059 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:41:56.670454 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:41:56.673231 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:41:56.673470 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.673722 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:41:56.673898 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:41:56.723229 12668 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:41:56.793840 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.3:44567
I20260807 05:41:56.793942 12749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.3:44567 every 8 connection(s)
I20260807 05:41:56.829576 12750 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
I20260807 05:41:56.829995 12750 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.831353 12750 heartbeater.cc:511] Master 127.10.111.60:42959 requested a full tablet report, sending...
I20260807 05:41:56.834826 12405 ts_manager.cc:194] Registered new tserver with Master: 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:56.834973 12751 heartbeater.cc:347] Connected to a master server at 127.10.111.61:34761
I20260807 05:41:56.835407 12751 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.836771 12751 heartbeater.cc:511] Master 127.10.111.61:34761 requested a full tablet report, sending...
I20260807 05:41:56.839223 12336 ts_manager.cc:194] Registered new tserver with Master: 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:56.842886 12753 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40189
I20260807 05:41:56.843312 12753 heartbeater.cc:464] Registering TS with master...
I20260807 05:41:56.844152 12753 heartbeater.cc:511] Master 127.10.111.62:40189 requested a full tablet report, sending...
I20260807 05:41:56.846676 12258 ts_manager.cc:194] Registered new tserver with Master: 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:56.847186 10684 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.030723497s
I20260807 05:41:56.849046 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34106
I20260807 05:41:56.899613 12258 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260807 05:41:56.901333 12258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:41:56.952343 12753 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:41:57.035694 12710 tablet_service.cc:1467] Processing CreateTablet for tablet eef56a6c70f64d41bee114c1bead379f (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.036418 12708 tablet_service.cc:1467] Processing CreateTablet for tablet 9c80213b4f9445dbac289b9f6d29fef9 (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.039628 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef56a6c70f64d41bee114c1bead379f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.041167 12709 tablet_service.cc:1467] Processing CreateTablet for tablet 5964010640c94067bdd81a066b40c0ac (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.037138 12542 tablet_service.cc:1467] Processing CreateTablet for tablet eef56a6c70f64d41bee114c1bead379f (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.046453 12540 tablet_service.cc:1467] Processing CreateTablet for tablet 9c80213b4f9445dbac289b9f6d29fef9 (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.043026 12541 tablet_service.cc:1467] Processing CreateTablet for tablet 5964010640c94067bdd81a066b40c0ac (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.044661 12625 tablet_service.cc:1467] Processing CreateTablet for tablet 5964010640c94067bdd81a066b40c0ac (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.051718 12708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c80213b4f9445dbac289b9f6d29fef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.057437 12625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5964010640c94067bdd81a066b40c0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.058104 12626 tablet_service.cc:1467] Processing CreateTablet for tablet eef56a6c70f64d41bee114c1bead379f (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.061822 12709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5964010640c94067bdd81a066b40c0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.060276 12542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef56a6c70f64d41bee114c1bead379f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.062124 12626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef56a6c70f64d41bee114c1bead379f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.062639 12624 tablet_service.cc:1467] Processing CreateTablet for tablet 9c80213b4f9445dbac289b9f6d29fef9 (DEFAULT_TABLE table=test-table [id=3c8b3a3f7c2047079fbd837048ec0354]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 05:41:57.067907 12540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c80213b4f9445dbac289b9f6d29fef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.069762 12541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5964010640c94067bdd81a066b40c0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.070683 12624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c80213b4f9445dbac289b9f6d29fef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:41:57.114598 12780 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: Bootstrap starting.
I20260807 05:41:57.129594 12780 tablet_bootstrap.cc:654] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.132711 12479 catalog_manager.cc:1259] Loaded cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:41:57.134122 12479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: loading cluster ID for follower catalog manager: success
I20260807 05:41:57.140681 12474 catalog_manager.cc:1259] Loaded cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:41:57.141139 12474 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: loading cluster ID for follower catalog manager: success
I20260807 05:41:57.140043 12781 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: Bootstrap starting.
I20260807 05:41:57.155346 12474 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: acquiring CA information for follower catalog manager: success
I20260807 05:41:57.158541 12479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: acquiring CA information for follower catalog manager: success
I20260807 05:41:57.162297 12474 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:41:57.163940 12479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:41:57.168982 12781 tablet_bootstrap.cc:654] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.142714 12782 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Bootstrap starting.
I20260807 05:41:57.179625 12780 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: No bootstrap required, opened a new log
I20260807 05:41:57.180298 12780 ts_tablet_manager.cc:1397] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent bootstrapping tablet: real 0.071s user 0.021s sys 0.018s
I20260807 05:41:57.183266 12782 tablet_bootstrap.cc:654] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.185959 12780 raft_consensus.cc:348] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.186992 12780 raft_consensus.cc:374] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.187470 12780 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.188369 12780 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.195124 12780 ts_tablet_manager.cc:1428] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 05:41:57.195065 12782 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: No bootstrap required, opened a new log
I20260807 05:41:57.196346 12780 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: Bootstrap starting.
I20260807 05:41:57.195796 12782 ts_tablet_manager.cc:1397] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.012s
I20260807 05:41:57.211355 12782 raft_consensus.cc:348] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.212574 12782 raft_consensus.cc:374] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.213109 12782 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.215059 12782 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.222738 12781 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: No bootstrap required, opened a new log
I20260807 05:41:57.223423 12781 ts_tablet_manager.cc:1397] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: Time spent bootstrapping tablet: real 0.085s user 0.015s sys 0.016s
I20260807 05:41:57.225040 12780 tablet_bootstrap.cc:654] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.227797 12781 raft_consensus.cc:348] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.228785 12781 raft_consensus.cc:374] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.229166 12781 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.231176 12786 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.233119 12786 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.234575 12781 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.239774 12788 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.245039 12786 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:41:57.240711 12788 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.295751 12782 ts_tablet_manager.cc:1428] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Time spent starting tablet: real 0.089s user 0.016s sys 0.029s
I20260807 05:41:57.298477 12788 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
W20260807 05:41:57.297116 12759 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.10.111.62:40189, user_credentials={real_user=slave}} blocked reactor thread for 52048.2us
I20260807 05:41:57.299572 12782 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: Bootstrap starting.
I20260807 05:41:57.349138 12781 ts_tablet_manager.cc:1428] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: Time spent starting tablet: real 0.125s user 0.018s sys 0.013s
I20260807 05:41:57.351275 12781 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: Bootstrap starting.
I20260807 05:41:57.352710 12791 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.353472 12791 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.356343 12780 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: No bootstrap required, opened a new log
I20260807 05:41:57.363898 12780 ts_tablet_manager.cc:1397] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent bootstrapping tablet: real 0.168s user 0.017s sys 0.024s
I20260807 05:41:57.360599 12782 tablet_bootstrap.cc:654] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.374500 12791 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.373430 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
W20260807 05:41:57.443416 12685 leader_election.cc:341] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.449568 12637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.450476 12637 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 0.
I20260807 05:41:57.462569 12781 tablet_bootstrap.cc:654] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.463577 12780 raft_consensus.cc:348] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.471959 12782 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: No bootstrap required, opened a new log
I20260807 05:41:57.472666 12782 ts_tablet_manager.cc:1397] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: Time spent bootstrapping tablet: real 0.175s user 0.009s sys 0.025s
I20260807 05:41:57.470468 12780 raft_consensus.cc:374] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.479439 12782 raft_consensus.cc:348] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.480243 12782 raft_consensus.cc:374] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.480517 12782 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.481523 12782 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.483502 12782 ts_tablet_manager.cc:1428] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: Time spent starting tablet: real 0.010s user 0.003s sys 0.001s
I20260807 05:41:57.475853 12780 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.485170 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
W20260807 05:41:57.490043 12601 leader_election.cc:341] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.492947 12782 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Bootstrap starting.
I20260807 05:41:57.486511 12780 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.499754 12781 tablet_bootstrap.cc:492] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: No bootstrap required, opened a new log
I20260807 05:41:57.500408 12781 ts_tablet_manager.cc:1397] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: Time spent bootstrapping tablet: real 0.149s user 0.013s sys 0.001s
I20260807 05:41:57.504911 12781 raft_consensus.cc:348] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.506582 12781 raft_consensus.cc:374] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.506858 12781 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.507622 12781 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.508796 12782 tablet_bootstrap.cc:654] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.512784 12781 ts_tablet_manager.cc:1428] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260807 05:41:57.513413 12786 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.514186 12786 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.519074 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
W20260807 05:41:57.520509 12685 leader_election.cc:341] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.521566 12786 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:57.522675 12781 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: Bootstrap starting.
I20260807 05:41:57.520602 12780 ts_tablet_manager.cc:1428] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent starting tablet: real 0.156s user 0.003s sys 0.008s
I20260807 05:41:57.524569 12780 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: Bootstrap starting.
I20260807 05:41:57.530292 12637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.531243 12637 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 0.
I20260807 05:41:57.539263 12781 tablet_bootstrap.cc:654] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.543740 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.548063 12637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.549172 12637 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 0.
I20260807 05:41:57.552342 12506 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters:
I20260807 05:41:57.554608 12794 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:57.555124 12794 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:57.555631 12794 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.561645 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:57.561789 12682 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:57.563054 12722 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 0.
I20260807 05:41:57.563623 12786 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:57.563988 12786 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:57.564546 12786 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.566054 12780 tablet_bootstrap.cc:654] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: Neither blocks nor log segments found. Creating new log.
I20260807 05:41:57.565747 12599 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
W20260807 05:41:57.564419 12682 leader_election.cc:341] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.568599 12682 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:57.568914 12790 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:57.569451 12790 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:57.570534 12787 raft_consensus.cc:2726] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:41:57.570952 12791 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.570051 12790 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.571533 12791 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.578548 12637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.571969 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:57.578012 12782 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: No bootstrap required, opened a new log
I20260807 05:41:57.581377 12782 ts_tablet_manager.cc:1397] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Time spent bootstrapping tablet: real 0.089s user 0.022s sys 0.002s
W20260807 05:41:57.582144 12507 leader_election.cc:341] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.578598 12791 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.581321 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:57.586715 12782 raft_consensus.cc:348] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.587697 12782 raft_consensus.cc:374] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.587817 12790 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.588263 12782 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.589406 12637 raft_consensus.cc:2379] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 in current term 1: Already voted for candidate 56379265ad0445b3af80f95324dc276a in this term.
I20260807 05:41:57.589196 12782 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.590590 12786 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.591995 12721 raft_consensus.cc:2379] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 in current term 1: Already voted for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in this term.
I20260807 05:41:57.593600 12786 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:41:57.595242 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:41:57.596148 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:57.596446 12782 ts_tablet_manager.cc:1428] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Time spent starting tablet: real 0.014s user 0.005s sys 0.004s
I20260807 05:41:57.596802 12552 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.595582 12794 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.601449 12790 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.603294 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:41:57.603910 12636 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.605587 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:57.608387 12780 tablet_bootstrap.cc:492] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: No bootstrap required, opened a new log
I20260807 05:41:57.608917 12780 ts_tablet_manager.cc:1397] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent bootstrapping tablet: real 0.085s user 0.011s sys 0.011s
I20260807 05:41:57.611572 12794 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.617676 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:57.610821 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:57.622857 12507 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a
I20260807 05:41:57.626138 12601 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56379265ad0445b3af80f95324dc276a; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:57.630544 12781 tablet_bootstrap.cc:492] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: No bootstrap required, opened a new log
I20260807 05:41:57.631570 12781 ts_tablet_manager.cc:1397] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.007s
W20260807 05:41:57.633638 12507 leader_election.cc:341] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567): Illegal state: must be running to vote when last-logged opid is not known
I20260807 05:41:57.636936 12781 raft_consensus.cc:348] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.637084 12780 raft_consensus.cc:348] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.638259 12781 raft_consensus.cc:374] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.638460 12780 raft_consensus.cc:374] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:41:57.638676 12790 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 05:41:57.638980 12780 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.640847 12780 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.647051 12552 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 1.
I20260807 05:41:57.638984 12781 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Initialized, Role: FOLLOWER
I20260807 05:41:57.651885 12780 ts_tablet_manager.cc:1428] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: Time spent starting tablet: real 0.042s user 0.010s sys 0.002s
I20260807 05:41:57.648396 12685 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:41:57.654860 12786 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:57.651489 12781 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.655721 12786 raft_consensus.cc:686] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:41:57.649449 12794 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:41:57.657371 12786 consensus_queue.cc:237] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.662171 12790 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.662743 12790 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.644716 12636 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 1.
I20260807 05:41:57.666426 12506 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1
I20260807 05:41:57.669895 12790 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.674659 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:57.675668 12722 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 0.
I20260807 05:41:57.677417 12599 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters:
I20260807 05:41:57.678514 12790 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:41:57.678926 12790 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:57.679395 12790 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.679980 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:57.681366 12552 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 0.
I20260807 05:41:57.687723 12791 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:57.687995 12781 ts_tablet_manager.cc:1428] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: Time spent starting tablet: real 0.056s user 0.008s sys 0.001s
I20260807 05:41:57.706748 12786 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:57.707492 12786 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.709959 12790 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.711421 12791 raft_consensus.cc:686] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 LEADER]: Becoming Leader. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:41:57.713985 12791 consensus_queue.cc:237] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.715801 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:57.717267 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:57.711288 12786 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:57.736368 12636 raft_consensus.cc:2379] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d8700bb6b514f88951a1f6db89bb9e1 in current term 1: Already voted for candidate e8f9fe91e3164e39bb3d65835859ba06 in this term.
I20260807 05:41:57.741961 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:57.742705 12722 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.748862 12790 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:57.753726 12722 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 1.
I20260807 05:41:57.754810 12599 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters:
I20260807 05:41:57.755959 12790 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:41:57.756450 12790 raft_consensus.cc:686] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 LEADER]: Becoming Leader. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:57.757364 12790 consensus_queue.cc:237] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:57.773180 12786 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:57.782081 12812 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 783781150ce54c45b48341c0f1386ebb)
I20260807 05:41:57.782759 12812 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:57.787236 12268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "783781150ce54c45b48341c0f1386ebb" is_pre_election: true
I20260807 05:41:57.793777 12812 leader_election.cc:288] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:41:57.822991 12817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 783781150ce54c45b48341c0f1386ebb)
I20260807 05:41:57.823781 12817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:57.827008 12817 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761)
I20260807 05:41:57.850770 12790 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:57.873565 12268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e2b5735fd7d470e866112a36570d269" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "783781150ce54c45b48341c0f1386ebb" is_pre_election: true
I20260807 05:41:57.874928 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e2b5735fd7d470e866112a36570d269" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" is_pre_election: true
I20260807 05:41:57.875605 12347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e2b5735fd7d470e866112a36570d269 in term 1.
I20260807 05:41:57.877085 12387 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e2b5735fd7d470e866112a36570d269, cd137409d1b9434e8fce87acd2f0f8e6; no voters: 783781150ce54c45b48341c0f1386ebb
I20260807 05:41:57.877990 12817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:41:57.878487 12817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 783781150ce54c45b48341c0f1386ebb)
I20260807 05:41:57.878983 12817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:57.896546 12416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269" is_pre_election: true
I20260807 05:41:57.897017 12817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:57.904758 12817 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 2 election: Requested vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761)
I20260807 05:41:57.910074 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e2b5735fd7d470e866112a36570d269" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
I20260807 05:41:57.911017 12347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:57.916518 12268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e2b5735fd7d470e866112a36570d269" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "783781150ce54c45b48341c0f1386ebb"
I20260807 05:41:57.921326 12318 leader_election.cc:302] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd137409d1b9434e8fce87acd2f0f8e6; no voters: 1e2b5735fd7d470e866112a36570d269, 783781150ce54c45b48341c0f1386ebb
I20260807 05:41:57.923574 12416 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Rejecting Update request from peer 783781150ce54c45b48341c0f1386ebb for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:57.931028 12347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e2b5735fd7d470e866112a36570d269 in term 2.
I20260807 05:41:57.931643 12812 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:41:57.933576 12387 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e2b5735fd7d470e866112a36570d269, cd137409d1b9434e8fce87acd2f0f8e6; no voters: 783781150ce54c45b48341c0f1386ebb
I20260807 05:41:57.935086 12817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:41:57.936635 12817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 LEADER]: Becoming Leader. State: Replica: 1e2b5735fd7d470e866112a36570d269, State: Running, Role: LEADER
I20260807 05:41:57.937646 12817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:57.971962 12346 raft_consensus.cc:1226] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Rejecting Update request from peer 783781150ce54c45b48341c0f1386ebb for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:57.972204 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:57.973016 12551 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:57.975481 12820 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 }, Status: INVALID_TERM, Last received: 1.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 05:41:57.977703 12820 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:57.978097 12820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 783781150ce54c45b48341c0f1386ebb, State: Running, Role: LEADER
I20260807 05:41:57.979810 12821 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 }, Status: INVALID_TERM, Last received: 1.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260807 05:41:57.982739 12551 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 1.
W20260807 05:41:57.993575 12552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48568 (request call id 3) took 281 ms (client timeout 268 ms). Trace:
W20260807 05:41:57.994019 12552 rpcz_store.cc:269] 0807 05:41:57.711840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 05:41:57.715375 (+ 3535us) service_pool.cc:227] Handling call
0807 05:41:57.993499 (+278124us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260807 05:41:58.003912 12416 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 LEADER]: Rejecting Update request from peer 783781150ce54c45b48341c0f1386ebb for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.006405 12773 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 }, Status: INVALID_TERM, Last received: 1.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 05:41:58.007537 12773 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Peer 1e2b5735fd7d470e866112a36570d269 log is divergent from this leader: its last log entry 2.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260807 05:41:58.020509 12346 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Refusing update from remote peer 1e2b5735fd7d470e866112a36570d269: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260807 05:41:58.022823 12817 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 05:41:58.028604 12790 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:58.029299 12790 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.032284 12790 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.031811 12822 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2b5735fd7d470e866112a36570d269. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.033232 12822 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:58.034165 12790 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:41:58.034950 12790 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.037417 12790 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.043756 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.044335 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.045039 12551 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 1.
I20260807 05:41:58.048704 12822 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.050103 12822 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:58.056563 12819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2b5735fd7d470e866112a36570d269. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.057579 12819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:58.057898 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:58.059437 12551 raft_consensus.cc:2362] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 1. Re-sending same reply.
I20260807 05:41:58.061113 12825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.062199 12825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:58.061923 12790 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.057385 12601 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters:
I20260807 05:41:58.065245 12790 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:41:58.065749 12787 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:41:58.066480 12787 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.067162 12828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:41:58.077247 12787 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:58.081174 12790 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:41:58.081686 12790 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.081739 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:58.086946 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:58.087616 12722 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 0.
I20260807 05:41:58.088594 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:58.093977 12794 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.094127 12790 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.097587 12790 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.099241 12346 raft_consensus.cc:1226] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Rejecting Update request from peer 783781150ce54c45b48341c0f1386ebb for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.098973 12820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:58.101063 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:58.102087 12552 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.112845 12552 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 2.
I20260807 05:41:58.114794 12601 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters:
I20260807 05:41:58.116245 12788 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:41:58.116950 12599 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:58.117007 12788 raft_consensus.cc:686] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Becoming Leader. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:58.119315 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:58.119213 12788 consensus_queue.cc:237] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.124951 12773 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 }, Status: INVALID_TERM, Last received: 1.6, Next index: 7, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 05:41:58.125679 12828 catalog_manager.cc:669] Loaded metadata for table test-table [id=3c8b3a3f7c2047079fbd837048ec0354]
I20260807 05:41:58.121418 12257 catalog_manager.cc:5495] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1). New cstate: current_term: 1 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:58.129300 12820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 1 FOLLOWER]: Advancing to term 2
W20260807 05:41:58.152328 12685 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.111.1:46737, user_credentials={real_user=slave}} blocked reactor thread for 59385.3us
E20260807 05:41:58.155643 12257 catalog_manager.cc:5650] Error updating tablets from e8f9fe91e3164e39bb3d65835859ba06: Illegal state: replica 783781150ce54c45b48341c0f1386ebb is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: true updated_tablets { tablet_id: "5964010640c94067bdd81a066b40c0ac" state: RUNNING consensus_state { current_term: 1 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } updated_tablets { tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } sequence_number: 14
I20260807 05:41:58.158402 12790 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:41:58.165504 12790 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:41:58.166079 12790 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.159929 12720 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:41:58.170001 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.170522 12828 tablet_loader.cc:96] loaded metadata for tablet 5964010640c94067bdd81a066b40c0ac (table test-table [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:41:58.174921 12787 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.175257 12685 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:58.188670 12787 raft_consensus.cc:2726] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:41:58.184216 12828 tablet_loader.cc:96] loaded metadata for tablet 9c80213b4f9445dbac289b9f6d29fef9 (table test-table [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:41:58.191437 12552 raft_consensus.cc:1226] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Rejecting Update request from peer 3d8700bb6b514f88951a1f6db89bb9e1 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 05:41:58.191787 12268 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Refusing update from remote peer 1e2b5735fd7d470e866112a36570d269: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260807 05:41:58.192867 12821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:58.193408 12787 consensus_queue.cc:1046] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.193455 12821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:41:58.194751 12786 raft_consensus.cc:3032] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:58.195227 12786 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:41:58.190986 12808 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.196203 12786 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.198138 12828 tablet_loader.cc:96] loaded metadata for tablet eef56a6c70f64d41bee114c1bead379f (table test-table [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:41:58.186571 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.197943 12786 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.202515 12416 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 LEADER]: Rejecting Update request from peer 783781150ce54c45b48341c0f1386ebb for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.204301 12828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 05:41:58.209667 12759 master_proxy_rpc.cc:203] Re-attempting IsCreateTableDone request to leader Master (127.10.111.62:40189)
I20260807 05:41:58.217758 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" is_pre_election: true
I20260807 05:41:58.218345 12821 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:41:58.220983 12819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 05:41:58.228348 12828 catalog_manager.cc:1259] Loaded cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:41:58.242396 12828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:41:58.222457 12416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269" is_pre_election: true
I20260807 05:41:58.186339 12790 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.245435 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb; no voters: 1e2b5735fd7d470e866112a36570d269, cd137409d1b9434e8fce87acd2f0f8e6
I20260807 05:41:58.246052 12636 raft_consensus.cc:1226] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Rejecting Update request from peer e8f9fe91e3164e39bb3d65835859ba06 for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.228356 12492 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:41:58.229106 12685 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:58.231045 12635 raft_consensus.cc:1226] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Rejecting Update request from peer 3d8700bb6b514f88951a1f6db89bb9e1 for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.249297 12808 consensus_queue.cc:1046] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.250340 12808 raft_consensus.cc:3032] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:58.250699 12808 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:41:58.251622 12808 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.253144 12808 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.255191 12790 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.250356 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:58.257444 12721 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Refusing update from remote peer 56379265ad0445b3af80f95324dc276a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.259102 12828 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:41:58.267730 12787 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:41:58.276831 12552 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Refusing update from remote peer 56379265ad0445b3af80f95324dc276a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.278990 12839 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:41:58.280872 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:58.270568 12791 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:41:58.282296 12791 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.286849 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:58.293346 12791 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.295059 12839 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:58.295451 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:58.297276 12720 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 1.
I20260807 05:41:58.299734 12507 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:41:58.301697 12791 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:41:58.302341 12791 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:41:58.309628 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2b5735fd7d470e866112a36570d269. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.310535 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:58.303279 12828 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Loaded TSK: 0
I20260807 05:41:58.310106 12791 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.318354 12828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:41:58.303824 12551 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 2.
I20260807 05:41:58.305954 12821 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 05:41:58.325667 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.326576 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:41:58.343771 12819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1e2b5735fd7d470e866112a36570d269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:41:58.344833 12819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: LEADER
I20260807 05:41:58.341730 12791 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.348385 12550 raft_consensus.cc:1226] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Rejecting Update request from peer 56379265ad0445b3af80f95324dc276a for earlier term 1. Current term is 2. Ops: []
I20260807 05:41:58.351017 12788 consensus_queue.cc:1046] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:58.357889 12834 raft_consensus.cc:3032] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 LEADER]: Stepping down as leader of term 1
I20260807 05:41:58.358336 12834 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:58.359506 12834 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.361647 12834 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.362401 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:41:58.366235 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:58.368155 12601 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1
I20260807 05:41:58.372797 12791 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 2 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:58.374606 12836 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:41:58.379743 12836 raft_consensus.cc:686] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Becoming Leader. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:58.380863 12836 consensus_queue.cc:237] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.390697 12635 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 2.
I20260807 05:41:58.392860 12506 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1
I20260807 05:41:58.394369 12791 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:41:58.394886 12791 raft_consensus.cc:686] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 LEADER]: Becoming Leader. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:41:58.395772 12791 consensus_queue.cc:237] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.401093 12404 catalog_manager.cc:5495] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 0 to 2, leader changed from <none> to 56379265ad0445b3af80f95324dc276a (127.10.111.2). New cstate: current_term: 2 leader_uuid: "56379265ad0445b3af80f95324dc276a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } } }
I20260807 05:41:58.414323 12405 catalog_manager.cc:5495] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a reported cstate change: term changed from 0 to 2, leader changed from <none> to 56379265ad0445b3af80f95324dc276a (127.10.111.2). New cstate: current_term: 2 leader_uuid: "56379265ad0445b3af80f95324dc276a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:58.440070 12581 heartbeater.cc:503] Master 127.10.111.60:42959 was elected leader, sending a full tablet report...
I20260807 05:41:58.450026 12664 heartbeater.cc:503] Master 127.10.111.60:42959 was elected leader, sending a full tablet report...
I20260807 05:41:58.454078 12404 catalog_manager.cc:5495] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 0 to 2, leader changed from <none> to e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1). New cstate: current_term: 2 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:58.491696 12720 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.495134 12552 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Refusing update from remote peer 56379265ad0445b3af80f95324dc276a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.497298 12836 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 05:41:58.498613 12721 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:41:58.500113 12720 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Refusing update from remote peer 56379265ad0445b3af80f95324dc276a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.500326 12635 raft_consensus.cc:1261] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.501106 12787 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:58.501638 12787 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.502486 12791 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:58.504060 12787 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:58.504381 12834 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 05:41:58.513439 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.519901 12750 heartbeater.cc:503] Master 127.10.111.60:42959 was elected leader, sending a full tablet report...
I20260807 05:41:58.522068 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:58.523727 12721 raft_consensus.cc:1261] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:41:58.528798 12794 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 05:41:58.533121 12787 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:58.533654 12787 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:58.533937 12682 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:58.536669 12786 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 05:41:58.542703 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:58.551419 12787 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:58.562973 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:58.564312 12682 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:58.565541 12787 raft_consensus.cc:2726] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 05:41:58.924568 12576 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7500 after lost signal to thread 10687
W20260807 05:41:58.925812 12576 debug-util.cc:398] Leaking SignalData structure 0x7b080008f300 after lost signal to thread 12298
W20260807 05:41:58.927145 12576 debug-util.cc:398] Leaking SignalData structure 0x7b080012f860 after lost signal to thread 12371
W20260807 05:41:58.928967 12576 debug-util.cc:398] Leaking SignalData structure 0x7b080023f040 after lost signal to thread 12445
W20260807 05:41:58.937883 12576 debug-util.cc:398] Leaking SignalData structure 0x7b08001b5e40 after lost signal to thread 12749
I20260807 05:41:59.376072 12552 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9"
dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
from {username='slave'} at 127.0.0.1:48600
I20260807 05:41:59.377400 12552 raft_consensus.cc:580] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 LEADER]: Received request to step down
I20260807 05:41:59.377820 12552 raft_consensus.cc:3032] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 05:41:59.378132 12552 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:41:59.379097 12552 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.381544 12552 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:59.538805 12834 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:41:59.539502 12834 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.541965 12834 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:59.543066 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:59.543521 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:59.545168 12552 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 3.
I20260807 05:41:59.546404 12601 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1
I20260807 05:41:59.547183 12834 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:41:59.547636 12834 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:41:59.548041 12834 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:59.553676 12834 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.555269 12834 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 3 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:59.556282 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:41:59.572700 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:59.575397 12552 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 3.
I20260807 05:41:59.577212 12601 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1
I20260807 05:41:59.578142 12834 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:41:59.578856 12834 raft_consensus.cc:686] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 LEADER]: Becoming Leader. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:59.579977 12834 consensus_queue.cc:237] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.596616 12867 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:41:59.603549 12867 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.609609 12867 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:41:59.610610 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:41:59.612772 12552 raft_consensus.cc:2379] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d8700bb6b514f88951a1f6db89bb9e1 in current term 3: Already voted for candidate 56379265ad0445b3af80f95324dc276a in this term.
I20260807 05:41:59.614233 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:59.616503 12682 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:41:59.620889 12404 catalog_manager.cc:5495] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a reported cstate change: term changed from 2 to 3, leader changed from e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1) to 56379265ad0445b3af80f95324dc276a (127.10.111.2). New cstate: current_term: 3 leader_uuid: "56379265ad0445b3af80f95324dc276a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:59.623291 12867 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:59.631445 12867 raft_consensus.cc:2726] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 05:41:59.649087 12635 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9"
dest_uuid: "56379265ad0445b3af80f95324dc276a"
from {username='slave'} at 127.0.0.1:43752
I20260807 05:41:59.650043 12635 raft_consensus.cc:580] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 LEADER]: Received request to step down
I20260807 05:41:59.650488 12635 raft_consensus.cc:3032] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 LEADER]: Stepping down as leader of term 3
I20260807 05:41:59.650854 12635 raft_consensus.cc:729] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:59.651829 12635 consensus_queue.cc:260] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.654102 12635 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:41:59.838689 12808 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:41:59.839251 12808 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.841276 12808 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:59.841933 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:41:59.842344 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:41:59.842732 12635 raft_consensus.cc:2396] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260807 05:41:59.843070 12721 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 3.
I20260807 05:41:59.844333 12507 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:41:59.845044 12808 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 05:41:59.845444 12808 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:41:59.845775 12808 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:41:59.853250 12808 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.855341 12808 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:41:59.856259 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:41:59.856561 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c80213b4f9445dbac289b9f6d29fef9" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:41:59.857173 12721 raft_consensus.cc:3037] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:41:59.857250 12635 raft_consensus.cc:2396] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260807 05:41:59.870152 12635 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac"
dest_uuid: "56379265ad0445b3af80f95324dc276a"
from {username='slave'} at 127.0.0.1:43752
I20260807 05:41:59.870759 12635 raft_consensus.cc:580] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Received request to step down
I20260807 05:41:59.873066 12635 raft_consensus.cc:3032] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Stepping down as leader of term 2
I20260807 05:41:59.873335 12635 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:41:59.874082 12635 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.875172 12635 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:41:59.880226 12721 raft_consensus.cc:2454] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 4.
I20260807 05:41:59.882718 12507 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:41:59.883952 12808 raft_consensus.cc:2781] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 05:41:59.886180 12808 raft_consensus.cc:686] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Becoming Leader. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:41:59.887128 12808 consensus_queue.cc:237] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:41:59.897408 12404 catalog_manager.cc:5495] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 3 to 4, leader changed from 56379265ad0445b3af80f95324dc276a (127.10.111.2) to e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1). New cstate: current_term: 4 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:41:59.977216 12635 raft_consensus.cc:1261] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260807 05:41:59.977682 12635 pending_rounds.cc:76] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Aborting all ops after (but not including) 3
I20260807 05:41:59.977950 12635 pending_rounds.cc:97] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260807 05:41:59.978271 12635 raft_consensus.cc:2866] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 05:41:59.979768 12808 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:42:00.009665 12721 raft_consensus.cc:1261] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 05:42:00.016772 12880 consensus_queue.cc:1035] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 05:42:00.029733 12867 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.030346 12867 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.032385 12867 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:42:00.033203 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:42:00.036418 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.037242 12635 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 3.
I20260807 05:42:00.038400 12682 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:00.039657 12867 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:42:00.040038 12867 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.040436 12867 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:00.049522 12880 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.050760 12880 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.054757 12880 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.055563 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.056336 12635 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 3.
I20260807 05:42:00.048702 12867 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.057422 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.058274 12721 raft_consensus.cc:2379] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 in current term 3: Already voted for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in this term.
I20260807 05:42:00.058183 12506 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06; no voters:
I20260807 05:42:00.059530 12867 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:42:00.059739 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:42:00.060115 12880 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:42:00.060624 12880 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.061033 12880 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:00.060830 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:42:00.067812 12636 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 3.
I20260807 05:42:00.067839 12880 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.069211 12552 raft_consensus.cc:2379] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d8700bb6b514f88951a1f6db89bb9e1 in current term 3: Already voted for candidate e8f9fe91e3164e39bb3d65835859ba06 in this term.
I20260807 05:42:00.069085 12682 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters:
I20260807 05:42:00.070037 12867 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:42:00.071640 12867 raft_consensus.cc:686] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Becoming Leader. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:42:00.075098 12880 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.075618 12867 consensus_queue.cc:237] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.076231 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:42:00.136952 12636 raft_consensus.cc:2379] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 in current term 3: Already voted for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in this term.
I20260807 05:42:00.077394 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:42:00.089316 12635 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f"
dest_uuid: "56379265ad0445b3af80f95324dc276a"
from {username='slave'} at 127.0.0.1:43752
I20260807 05:42:00.139775 12635 raft_consensus.cc:580] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Received request to step down
I20260807 05:42:00.140182 12635 raft_consensus.cc:3032] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Stepping down as leader of term 2
I20260807 05:42:00.140520 12635 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:42:00.141429 12635 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.142253 12507 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a
I20260807 05:42:00.142740 12635 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:00.143127 12880 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 05:42:00.154711 12405 catalog_manager.cc:5495] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 reported cstate change: term changed from 2 to 3, leader changed from 56379265ad0445b3af80f95324dc276a (127.10.111.2) to 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3). New cstate: current_term: 3 leader_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: HEALTHY } } }
I20260807 05:42:00.244387 12552 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Refusing update from remote peer 3d8700bb6b514f88951a1f6db89bb9e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 05:42:00.248835 12867 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 05:42:00.262287 12635 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Refusing update from remote peer 3d8700bb6b514f88951a1f6db89bb9e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 05:42:00.264930 12887 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 05:42:00.284111 12889 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.284848 12889 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.293397 12889 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:42:00.293977 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.295326 12635 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 3.
I20260807 05:42:00.296882 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:42:00.297884 12682 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters:
I20260807 05:42:00.307379 12890 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:42:00.307968 12890 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.308476 12890 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:00.319031 12890 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.322789 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06"
I20260807 05:42:00.323480 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:42:00.332445 12636 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in term 3.
I20260807 05:42:00.333573 12890 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:42:00.334419 12682 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:00.334940 12808 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56379265ad0445b3af80f95324dc276a)
I20260807 05:42:00.335500 12808 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.335918 12890 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:42:00.336753 12890 raft_consensus.cc:686] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Becoming Leader. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:42:00.337899 12890 consensus_queue.cc:237] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.340227 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.342160 12808 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.344357 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.345384 12636 raft_consensus.cc:2379] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8f9fe91e3164e39bb3d65835859ba06 in current term 3: Already voted for candidate 3d8700bb6b514f88951a1f6db89bb9e1 in this term.
I20260807 05:42:00.349999 12506 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8f9fe91e3164e39bb3d65835859ba06; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a
I20260807 05:42:00.361135 12405 catalog_manager.cc:5495] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 reported cstate change: term changed from 2 to 3, leader changed from 56379265ad0445b3af80f95324dc276a (127.10.111.2) to 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3). New cstate: current_term: 3 leader_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: HEALTHY } } }
I20260807 05:42:00.364100 12808 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:00.373047 12808 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 05:42:00.439815 12720 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac"
dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
from {username='slave'} at 127.0.0.1:59138
I20260807 05:42:00.440670 12720 raft_consensus.cc:580] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Received request to step down
I20260807 05:42:00.441253 12720 raft_consensus.cc:3032] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 05:42:00.441629 12720 raft_consensus.cc:729] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:42:00.442574 12720 consensus_queue.cc:260] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.444317 12720 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.463263 12552 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Refusing update from remote peer 3d8700bb6b514f88951a1f6db89bb9e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 05:42:00.465409 12636 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Refusing update from remote peer 3d8700bb6b514f88951a1f6db89bb9e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 05:42:00.466424 12887 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:42:00.467358 12889 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:42:00.586724 12882 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.587196 12882 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.588981 12882 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.589987 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.589977 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.590588 12720 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 4.
I20260807 05:42:00.592025 12507 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:42:00.593119 12882 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 05:42:00.593449 12882 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.593786 12882 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.599898 12882 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.601621 12882 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.602394 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:42:00.602988 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:42:00.615260 12720 raft_consensus.cc:2454] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 4.
I20260807 05:42:00.616561 12507 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:42:00.617408 12882 raft_consensus.cc:2781] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 05:42:00.618414 12882 raft_consensus.cc:686] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Becoming Leader. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:42:00.619246 12882 consensus_queue.cc:237] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.624576 12834 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.625072 12834 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.627164 12834 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.629930 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:42:00.629951 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.633440 12405 catalog_manager.cc:5495] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 3 to 4, leader changed from 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3) to e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1). New cstate: current_term: 4 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:42:00.634320 12720 raft_consensus.cc:2379] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56379265ad0445b3af80f95324dc276a in current term 4: Already voted for candidate e8f9fe91e3164e39bb3d65835859ba06 in this term.
I20260807 05:42:00.635798 12599 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56379265ad0445b3af80f95324dc276a; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:00.638605 12834 raft_consensus.cc:3037] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.643431 12834 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 05:42:00.668399 12721 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f"
dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
from {username='slave'} at 127.0.0.1:59138
I20260807 05:42:00.669081 12721 raft_consensus.cc:580] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Received request to step down
I20260807 05:42:00.669384 12721 raft_consensus.cc:3032] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 05:42:00.669667 12721 raft_consensus.cc:729] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d8700bb6b514f88951a1f6db89bb9e1, State: Running, Role: LEADER
I20260807 05:42:00.670382 12721 consensus_queue.cc:260] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.671741 12721 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.702714 12636 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 05:42:00.704195 12882 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 05:42:00.712857 12721 raft_consensus.cc:1261] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 05:42:00.716298 12882 consensus_queue.cc:1035] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 05:42:00.774297 12897 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.774806 12897 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.776373 12897 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.777495 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.777339 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:00.778120 12721 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 4.
I20260807 05:42:00.779119 12507 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters:
I20260807 05:42:00.779892 12897 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 05:42:00.780274 12897 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.780663 12897 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.788617 12897 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.790654 12897 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Requested vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.791460 12636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a"
I20260807 05:42:00.791898 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "e8f9fe91e3164e39bb3d65835859ba06" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:42:00.799383 12721 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f9fe91e3164e39bb3d65835859ba06 in term 4.
I20260807 05:42:00.800493 12507 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06; no voters: 56379265ad0445b3af80f95324dc276a
I20260807 05:42:00.801275 12897 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 05:42:00.801808 12897 raft_consensus.cc:686] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Becoming Leader. State: Replica: e8f9fe91e3164e39bb3d65835859ba06, State: Running, Role: LEADER
I20260807 05:42:00.802795 12897 consensus_queue.cc:237] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.810021 12834 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d8700bb6b514f88951a1f6db89bb9e1)
I20260807 05:42:00.810573 12834 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:00.812623 12834 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:00.815008 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:42:00.818471 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:00.819186 12721 raft_consensus.cc:2379] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56379265ad0445b3af80f95324dc276a in current term 4: Already voted for candidate e8f9fe91e3164e39bb3d65835859ba06 in this term.
I20260807 05:42:00.820012 12599 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56379265ad0445b3af80f95324dc276a; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:00.820749 12834 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:00.820912 12405 catalog_manager.cc:5495] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 reported cstate change: term changed from 3 to 4, leader changed from 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3) to e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1). New cstate: current_term: 4 leader_uuid: "e8f9fe91e3164e39bb3d65835859ba06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } health_report { overall_health: UNKNOWN } } }
I20260807 05:42:00.826529 12834 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 05:42:00.882725 12404 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37850:
table { table_id: "3c8b3a3f7c2047079fbd837048ec0354" table_name: "test-table" } new_table_name: "NewTableName"
I20260807 05:42:00.898486 12721 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 05:42:00.902827 12808 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 05:42:00.914057 12636 raft_consensus.cc:1261] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Refusing update from remote peer e8f9fe91e3164e39bb3d65835859ba06: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 05:42:00.916213 12808 consensus_queue.cc:1035] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 05:42:00.927173 12401 catalog_manager.cc:5587] TS 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567) does not have the latest schema for tablet eef56a6c70f64d41bee114c1bead379f (table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354]). Expected version 1 got 0
I20260807 05:42:00.958351 10684 master.cc:537] Master@127.10.111.60:42959 shutting down...
I20260807 05:42:01.031554 12907 tablet.cc:1722] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.033905 12404 catalog_manager.cc:5587] TS 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769) does not have the latest schema for tablet eef56a6c70f64d41bee114c1bead379f (table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354]). Expected version 1 got 0
I20260807 05:42:01.060616 12907 tablet.cc:1722] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.075826 12907 tablet.cc:1722] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.083170 12915 tablet.cc:1722] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.099851 12913 tablet.cc:1722] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.103565 12914 tablet.cc:1722] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.124485 12915 tablet.cc:1722] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.148502 12910 tablet.cc:1722] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.155032 12911 tablet.cc:1722] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 05:42:01.183185 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:42:01.184652 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:01.185271 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: stopping tablet replica
I20260807 05:42:01.219769 10684 master.cc:552] Master@127.10.111.60:42959 shutdown complete.
I20260807 05:42:01.244067 12889 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:01.244776 12889 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:01.246464 12889 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:42:01.247262 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "56379265ad0445b3af80f95324dc276a" is_pre_election: true
I20260807 05:42:01.247699 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e8f9fe91e3164e39bb3d65835859ba06" is_pre_election: true
I20260807 05:42:01.249275 12685 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:01.250224 12889 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 05:42:01.319944 12919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1e2b5735fd7d470e866112a36570d269)
I20260807 05:42:01.320416 12919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.322150 12919 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:42:01.323504 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" is_pre_election: true
I20260807 05:42:01.324353 12346 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 2.
I20260807 05:42:01.326666 12920 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1e2b5735fd7d470e866112a36570d269)
I20260807 05:42:01.327127 12920 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.333284 12268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "783781150ce54c45b48341c0f1386ebb" is_pre_election: true
I20260807 05:42:01.334054 12268 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd137409d1b9434e8fce87acd2f0f8e6 in term 2.
I20260807 05:42:01.334898 12920 leader_election.cc:288] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:42:01.335901 12318 leader_election.cc:302] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6; no voters:
W20260807 05:42:01.336851 12318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260807 05:42:01.336867 12920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:42:01.337800 12920 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1e2b5735fd7d470e866112a36570d269)
I20260807 05:42:01.338483 12920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:01.347939 12920 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.350797 12920 leader_election.cc:288] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 election: Requested vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:42:01.352007 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6; no voters:
I20260807 05:42:01.352926 12919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 05:42:01.353354 12919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1e2b5735fd7d470e866112a36570d269)
I20260807 05:42:01.353787 12919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:42:01.354270 12268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "783781150ce54c45b48341c0f1386ebb"
W20260807 05:42:01.359189 12318 leader_election.cc:334] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959): Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
W20260807 05:42:01.359925 12318 leader_election.cc:334] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959): Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:42:01.362713 12919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.363921 12268 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd137409d1b9434e8fce87acd2f0f8e6 in current term 3: Already voted for candidate 783781150ce54c45b48341c0f1386ebb in this term.
I20260807 05:42:01.364917 12919 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 election: Requested vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
W20260807 05:42:01.365584 12242 leader_election.cc:334] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959): Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:42:01.367540 12318 leader_election.cc:302] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd137409d1b9434e8fce87acd2f0f8e6; no voters: 1e2b5735fd7d470e866112a36570d269, 783781150ce54c45b48341c0f1386ebb
I20260807 05:42:01.367686 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
I20260807 05:42:01.368561 12346 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 783781150ce54c45b48341c0f1386ebb in current term 3: Already voted for candidate cd137409d1b9434e8fce87acd2f0f8e6 in this term.
I20260807 05:42:01.369666 12920 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 05:42:01.372720 12242 leader_election.cc:334] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959): Network error: Client connection negotiation failed: client connection to 127.10.111.60:42959: connect: Connection refused (error 111)
I20260807 05:42:01.373286 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb; no voters: 1e2b5735fd7d470e866112a36570d269, cd137409d1b9434e8fce87acd2f0f8e6
I20260807 05:42:01.374708 12919 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 05:42:01.441891 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:01.448866 12925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:01.449216 12927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:01.454115 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:01.454809 12929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:01.456125 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:01.456453 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:01.456653 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081321456636 us; error 0 us; skew 500 ppm
I20260807 05:42:01.457283 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:01.463161 10684 webserver.cc:460] Webserver started at http://127.10.111.60:42695/ using document root <none> and password file <none>
I20260807 05:42:01.463740 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:01.463934 10684 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 05:42:01.468456 10684 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:42:01.472151 12934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:01.473275 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:01.473641 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
uuid: "1e2b5735fd7d470e866112a36570d269"
format_stamp: "Formatted at 2026-08-07 05:41:55 on dist-test-slave-qg60"
I20260807 05:42:01.474170 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:01.513515 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:01.514812 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:01.569823 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.60:42959
I20260807 05:42:01.569980 12989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.60:42959 every 8 connection(s)
I20260807 05:42:01.585855 12990 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:42:01.594596 12990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Bootstrap starting.
I20260807 05:42:01.600795 12581 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
I20260807 05:42:01.620476 12664 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
I20260807 05:42:01.641732 12919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:42:01.642288 12919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.644518 12919 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:42:01.648790 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" is_pre_election: true
I20260807 05:42:01.649500 12346 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 3.
I20260807 05:42:01.650826 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6; no voters:
I20260807 05:42:01.651870 12919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 05:42:01.652590 12919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:42:01.653450 12919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:01.666831 12750 heartbeater.cc:347] Connected to a master server at 127.10.111.60:42959
I20260807 05:42:01.667280 12919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.677067 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "cd137409d1b9434e8fce87acd2f0f8e6"
I20260807 05:42:01.677731 12346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 3 FOLLOWER]: Advancing to term 4
I20260807 05:42:01.691305 12346 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 4.
I20260807 05:42:01.693599 12990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:01.694834 12990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: Bootstrap complete.
I20260807 05:42:01.697409 12242 leader_election.cc:302] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6; no voters:
I20260807 05:42:01.698556 12919 leader_election.cc:288] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [CANDIDATE]: Term 4 election: Requested vote from peers cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761), 1e2b5735fd7d470e866112a36570d269 (127.10.111.60:42959)
I20260807 05:42:01.698236 12990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.699832 12990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2b5735fd7d470e866112a36570d269, State: Initialized, Role: FOLLOWER
I20260807 05:42:01.700568 12990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.704838 12990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:01.705297 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.706342 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:42:01.707001 12919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 4 FOLLOWER]: Leader election won for term 4
I20260807 05:42:01.707494 12919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 4 LEADER]: Becoming Leader. State: Replica: 783781150ce54c45b48341c0f1386ebb, State: Running, Role: LEADER
I20260807 05:42:01.708257 12919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:01.713020 12919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.714075 12919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:01.727279 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269" is_pre_election: true
I20260807 05:42:01.728173 12963 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 2.
I20260807 05:42:01.729086 12998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:01.729602 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "783781150ce54c45b48341c0f1386ebb" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "1e2b5735fd7d470e866112a36570d269"
I20260807 05:42:01.730517 12962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 2 FOLLOWER]: Advancing to term 4
I20260807 05:42:01.741298 12962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783781150ce54c45b48341c0f1386ebb in term 4.
I20260807 05:42:01.743320 12998 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354]
I20260807 05:42:01.751412 12998 tablet_loader.cc:96] loaded metadata for tablet 5964010640c94067bdd81a066b40c0ac (table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:42:01.753075 12998 tablet_loader.cc:96] loaded metadata for tablet 9c80213b4f9445dbac289b9f6d29fef9 (table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:42:01.754679 12998 tablet_loader.cc:96] loaded metadata for tablet eef56a6c70f64d41bee114c1bead379f (table NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354])
I20260807 05:42:01.756395 12998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:01.758275 13008 catalog_manager.cc:1259] Loaded cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:42:01.758553 13008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: loading cluster ID for follower catalog manager: success
I20260807 05:42:01.761790 12998 catalog_manager.cc:1259] Loaded cluster ID: adb3976b94624aab913db59e7e889397
I20260807 05:42:01.762120 12998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:01.764415 13008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: acquiring CA information for follower catalog manager: success
I20260807 05:42:01.770059 13008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:42:01.770752 12998 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:01.776299 12998 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: Loaded TSK: 0
I20260807 05:42:01.777770 12998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:01.802341 12952 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" instance_seqno: 1786081316766239) as {username='slave'} at 127.0.0.1:38052; Asking this server to re-register.
I20260807 05:42:01.802341 12953 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "56379265ad0445b3af80f95324dc276a" instance_seqno: 1786081316511685) as {username='slave'} at 127.0.0.1:38042; Asking this server to re-register.
I20260807 05:42:01.804117 12750 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:01.806828 12664 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:01.809900 12952 ts_manager.cc:194] Registered new tserver with Master: 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:01.811441 12951 ts_manager.cc:194] Registered new tserver with Master: 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:42:01.812716 12346 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 4 FOLLOWER]: Refusing update from remote peer 783781150ce54c45b48341c0f1386ebb: Log matching property violated. Preceding OpId in replica: term: 2 index: 17. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20260807 05:42:01.815550 12919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260807 05:42:01.816224 12953 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" instance_seqno: 1786081316299378) as {username='slave'} at 127.0.0.1:38034; Asking this server to re-register.
I20260807 05:42:01.817631 12581 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:01.821472 12953 ts_manager.cc:194] Registered new tserver with Master: e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:42:01.821360 12920 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.821950 12920 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:42:01.824542 12962 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Refusing update from remote peer 783781150ce54c45b48341c0f1386ebb: Log matching property violated. Preceding OpId in replica: term: 2 index: 17. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20260807 05:42:01.827071 12919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260807 05:42:01.830837 12920 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.831624 12920 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:42:01.830832 12995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.832156 12995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:01.835994 13013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.836957 13013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:42:01.836529 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783781150ce54c45b48341c0f1386ebb. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.837453 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:42:01.897351 12753 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:42:01.900755 12668 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:42:01.913765 12583 heartbeater.cc:503] Master 127.10.111.62:40189 was elected leader, sending a full tablet report...
I20260807 05:42:01.920704 12919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "783781150ce54c45b48341c0f1386ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } } }
I20260807 05:42:01.921767 12919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:01.954267 12255 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: NewTableName [id=3c8b3a3f7c2047079fbd837048ec0354] alter complete (version 1)
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:783: Failure
Expected equality of these values:
live_row_count
Which is: 802
table_info->GetMetrics()->live_row_count->value()
Which is: 0
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_util.cc:395: Failure
Failed
Timed out waiting for assertion to pass.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:767: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:785: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 782, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:895: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260807 05:42:31.942401 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:31.986847 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:31.987651 10684 tablet_replica.cc:330] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06: stopping tablet replica
I20260807 05:42:31.988662 10684 raft_consensus.cc:2229] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Raft consensus shutting down.
I20260807 05:42:31.989816 10684 raft_consensus.cc:2258] T eef56a6c70f64d41bee114c1bead379f P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:31.992323 10684 tablet_replica.cc:330] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06: stopping tablet replica
I20260807 05:42:31.992902 10684 raft_consensus.cc:2229] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Raft consensus shutting down.
I20260807 05:42:31.993671 10684 raft_consensus.cc:2258] T 9c80213b4f9445dbac289b9f6d29fef9 P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:31.995968 10684 tablet_replica.cc:330] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06: stopping tablet replica
I20260807 05:42:31.996398 10684 raft_consensus.cc:2229] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 4 LEADER]: Raft consensus shutting down.
I20260807 05:42:31.997202 10684 raft_consensus.cc:2258] T 5964010640c94067bdd81a066b40c0ac P e8f9fe91e3164e39bb3d65835859ba06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.045521 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:32.063768 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:42:32.085075 13057 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.085803 13057 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:32.090351 13060 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.090937 13060 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
W20260807 05:42:32.097749 12682 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260807 05:42:32.098608 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:32.099818 12720 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 4.
I20260807 05:42:32.093099 13057 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:32.102666 12599 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters:
I20260807 05:42:32.096271 13060 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769), e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737)
I20260807 05:42:32.105086 13057 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 05:42:32.105895 13057 raft_consensus.cc:482] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Starting leader election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.106678 13057 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Advancing to term 5
I20260807 05:42:32.115212 13064 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.116312 13064 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
W20260807 05:42:32.118849 12601 leader_election.cc:334] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
I20260807 05:42:32.121569 13064 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
I20260807 05:42:32.121140 13057 raft_consensus.cc:504] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:32.122699 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5964010640c94067bdd81a066b40c0ac" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" is_pre_election: true
I20260807 05:42:32.124838 13057 leader_election.cc:288] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 election: Requested vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 3d8700bb6b514f88951a1f6db89bb9e1 (127.10.111.3:44567)
W20260807 05:42:32.124218 12682 leader_election.cc:334] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260807 05:42:32.126585 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef56a6c70f64d41bee114c1bead379f" candidate_uuid: "56379265ad0445b3af80f95324dc276a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d8700bb6b514f88951a1f6db89bb9e1"
I20260807 05:42:32.127157 12720 raft_consensus.cc:3037] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Advancing to term 5
W20260807 05:42:32.127507 12685 leader_election.cc:334] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
W20260807 05:42:32.127748 12601 leader_election.cc:334] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
I20260807 05:42:32.127995 12685 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:32.128180 12601 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56379265ad0445b3af80f95324dc276a; no voters: 3d8700bb6b514f88951a1f6db89bb9e1, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:32.129456 13057 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 05:42:32.131275 12601 leader_election.cc:334] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
I20260807 05:42:32.134146 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:32.136452 12720 raft_consensus.cc:2454] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56379265ad0445b3af80f95324dc276a in term 5.
I20260807 05:42:32.137142 10684 tablet_replica.cc:330] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a: stopping tablet replica
I20260807 05:42:32.137218 13060 raft_consensus.cc:2726] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 05:42:32.138005 10684 raft_consensus.cc:2229] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.138816 10684 raft_consensus.cc:2258] T 9c80213b4f9445dbac289b9f6d29fef9 P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.139968 13060 raft_consensus.cc:482] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.140295 13067 raft_consensus.cc:482] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8f9fe91e3164e39bb3d65835859ba06)
I20260807 05:42:32.140503 13060 raft_consensus.cc:504] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:32.140815 13067 raft_consensus.cc:504] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:32.143635 10684 tablet_replica.cc:330] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a: stopping tablet replica
I20260807 05:42:32.144330 13067 leader_election.cc:288] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:42:32.144685 10684 raft_consensus.cc:2229] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.145555 10684 raft_consensus.cc:2258] T 5964010640c94067bdd81a066b40c0ac P 56379265ad0445b3af80f95324dc276a [term 4 FOLLOWER]: Raft consensus is shut down!
W20260807 05:42:32.148102 12682 leader_election.cc:334] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260807 05:42:32.148306 13060 leader_election.cc:288] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737), 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769)
I20260807 05:42:32.150096 12599 leader_election.cc:302] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1, 56379265ad0445b3af80f95324dc276a; no voters: e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:32.150259 10684 tablet_replica.cc:330] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: stopping tablet replica
I20260807 05:42:32.150820 13057 raft_consensus.cc:2781] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 FOLLOWER]: Leader election won for term 5
W20260807 05:42:32.150938 12682 leader_election.cc:334] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56379265ad0445b3af80f95324dc276a (127.10.111.2:39769): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260807 05:42:32.151355 13057 raft_consensus.cc:686] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 LEADER]: Becoming Leader. State: Replica: 56379265ad0445b3af80f95324dc276a, State: Running, Role: LEADER
I20260807 05:42:32.152256 13057 consensus_queue.cc:237] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56379265ad0445b3af80f95324dc276a" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 39769 } } peers { permanent_uuid: "e8f9fe91e3164e39bb3d65835859ba06" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 46737 } } peers { permanent_uuid: "3d8700bb6b514f88951a1f6db89bb9e1" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 44567 } }
I20260807 05:42:32.155474 10684 raft_consensus.cc:2229] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 LEADER]: Raft consensus shutting down.
W20260807 05:42:32.156342 12685 leader_election.cc:334] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
I20260807 05:42:32.156929 12685 leader_election.cc:302] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
W20260807 05:42:32.158087 12685 leader_election.cc:334] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8f9fe91e3164e39bb3d65835859ba06 (127.10.111.1:46737): Network error: Client connection negotiation failed: client connection to 127.10.111.1:46737: connect: Connection refused (error 111)
I20260807 05:42:32.158079 13068 raft_consensus.cc:2726] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 05:42:32.158833 12685 leader_election.cc:302] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d8700bb6b514f88951a1f6db89bb9e1; no voters: 56379265ad0445b3af80f95324dc276a, e8f9fe91e3164e39bb3d65835859ba06
I20260807 05:42:32.159150 10684 pending_rounds.cc:62] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Trying to abort 1 pending ops.
I20260807 05:42:32.159474 10684 pending_rounds.cc:68] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a: Aborting op as it isn't in flight: id { term: 5 index: 7 } timestamp: 7315789218423037952 op_type: NO_OP noop_request { }
I20260807 05:42:32.159543 13068 raft_consensus.cc:2726] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 05:42:32.159970 10684 raft_consensus.cc:2866] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 05:42:32.160346 10684 raft_consensus.cc:2258] T eef56a6c70f64d41bee114c1bead379f P 56379265ad0445b3af80f95324dc276a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.185189 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:42:32.202064 10684 tablet_server.cc:178] TabletServer@127.10.111.3:0 shutting down...
I20260807 05:42:32.236475 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:32.237183 10684 tablet_replica.cc:330] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1: stopping tablet replica
I20260807 05:42:32.237793 10684 raft_consensus.cc:2229] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.238296 10684 raft_consensus.cc:2258] T 5964010640c94067bdd81a066b40c0ac P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.240662 10684 tablet_replica.cc:330] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1: stopping tablet replica
I20260807 05:42:32.241242 10684 raft_consensus.cc:2229] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.241699 10684 raft_consensus.cc:2258] T 9c80213b4f9445dbac289b9f6d29fef9 P 3d8700bb6b514f88951a1f6db89bb9e1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.244128 10684 tablet_replica.cc:330] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1: stopping tablet replica
I20260807 05:42:32.244623 10684 raft_consensus.cc:2229] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.245141 10684 raft_consensus.cc:2258] T eef56a6c70f64d41bee114c1bead379f P 3d8700bb6b514f88951a1f6db89bb9e1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.268535 10684 tablet_server.cc:194] TabletServer@127.10.111.3:0 shutdown complete.
I20260807 05:42:32.285668 10684 master.cc:537] Master@127.10.111.62:40189 shutting down...
I20260807 05:42:32.311940 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 4 LEADER]: Raft consensus shutting down.
I20260807 05:42:32.312958 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.313297 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 783781150ce54c45b48341c0f1386ebb: stopping tablet replica
I20260807 05:42:32.334105 10684 master.cc:552] Master@127.10.111.62:40189 shutdown complete.
I20260807 05:42:32.355868 10684 master.cc:537] Master@127.10.111.61:34761 shutting down...
I20260807 05:42:32.378932 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.379462 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.379741 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P cd137409d1b9434e8fce87acd2f0f8e6: stopping tablet replica
I20260807 05:42:32.398308 10684 master.cc:552] Master@127.10.111.61:34761 shutdown complete.
I20260807 05:42:32.416003 10684 master.cc:537] Master@127.10.111.60:42959 shutting down...
I20260807 05:42:32.432844 13070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 783781150ce54c45b48341c0f1386ebb)
I20260807 05:42:32.433336 13070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "783781150ce54c45b48341c0f1386ebb" member_type: VOTER last_known_addr { host: "127.10.111.62" port: 40189 } } peers { permanent_uuid: "cd137409d1b9434e8fce87acd2f0f8e6" member_type: VOTER last_known_addr { host: "127.10.111.61" port: 34761 } } peers { permanent_uuid: "1e2b5735fd7d470e866112a36570d269" member_type: VOTER last_known_addr { host: "127.10.111.60" port: 42959 } }
I20260807 05:42:32.438223 13070 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189), cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761)
I20260807 05:42:32.442440 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:32.443325 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:32.443786 10684 tablet_replica.cc:330] stopping tablet replica
W20260807 05:42:32.444667 12936 leader_election.cc:334] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 783781150ce54c45b48341c0f1386ebb (127.10.111.62:40189): Network error: Client connection negotiation failed: client connection to 127.10.111.62:40189: connect: Connection refused (error 111)
W20260807 05:42:32.445528 12936 leader_election.cc:334] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cd137409d1b9434e8fce87acd2f0f8e6 (127.10.111.61:34761): Network error: Client connection negotiation failed: client connection to 127.10.111.61:34761: connect: Connection refused (error 111)
I20260807 05:42:32.445907 12936 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e2b5735fd7d470e866112a36570d269; no voters: 783781150ce54c45b48341c0f1386ebb, cd137409d1b9434e8fce87acd2f0f8e6
W20260807 05:42:32.446563 12936 raft_consensus.cc:2670] T 00000000000000000000000000000000 P 1e2b5735fd7d470e866112a36570d269: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260807 05:42:32.466189 10684 master.cc:552] Master@127.10.111.60:42959 shutdown complete.
I20260807 05:42:32.493496 10684 test_util.cc:175] -----------------------------------------------
I20260807 05:42:32.493709 10684 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786081299864053-10684-0
[ FAILED ] TsTabletManagerITest.TestTableStats (36759 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260807 05:42:32.500233 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:35683
I20260807 05:42:32.501363 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:32.507427 13079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.507747 13080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.509451 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:32.509851 13082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.510807 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:32.511085 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:32.511247 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081352511215 us; error 0 us; skew 500 ppm
I20260807 05:42:32.511790 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:32.514346 10684 webserver.cc:460] Webserver started at http://127.10.111.62:44313/ using document root <none> and password file <none>
I20260807 05:42:32.514837 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:32.515041 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:32.515326 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:32.516623 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "a29dfa48c1564f2f8a800be45c3ccfed"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.521706 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:42:32.525225 13090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.526249 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:42:32.526503 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "a29dfa48c1564f2f8a800be45c3ccfed"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.526760 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:32.545953 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:32.547142 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:32.587734 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:35683
I20260807 05:42:32.587915 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:35683 every 8 connection(s)
I20260807 05:42:32.592214 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:32.603960 13147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: Bootstrap starting.
I20260807 05:42:32.609689 13147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:32.614495 13147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: No bootstrap required, opened a new log
I20260807 05:42:32.616942 13147 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER }
I20260807 05:42:32.617374 13147 raft_consensus.cc:374] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:32.617595 13147 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29dfa48c1564f2f8a800be45c3ccfed, State: Initialized, Role: FOLLOWER
I20260807 05:42:32.618230 13147 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER }
I20260807 05:42:32.618731 13147 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:32.618938 13147 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:32.619197 13147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:32.623873 13147 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER }
I20260807 05:42:32.624379 13147 leader_election.cc:302] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29dfa48c1564f2f8a800be45c3ccfed; no voters:
I20260807 05:42:32.625543 13147 leader_election.cc:288] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:32.625981 13150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:32.627614 13150 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 1 LEADER]: Becoming Leader. State: Replica: a29dfa48c1564f2f8a800be45c3ccfed, State: Running, Role: LEADER
I20260807 05:42:32.628355 13150 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER }
I20260807 05:42:32.629530 13147 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:32.631800 13151 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER } }
I20260807 05:42:32.631862 13152 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29dfa48c1564f2f8a800be45c3ccfed. Latest consensus state: current_term: 1 leader_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dfa48c1564f2f8a800be45c3ccfed" member_type: VOTER } }
I20260807 05:42:32.632665 13152 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:32.633906 13151 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:32.636446 13157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:32.641577 13157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:32.647411 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:32.651003 13157 catalog_manager.cc:1347] Generated new cluster ID: c5c528ed3b9f4b72bcb72e15a1036b1c
I20260807 05:42:32.651317 13157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:32.688609 13157 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:32.690544 13157 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:32.702469 13157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: Generated new TSK 0
I20260807 05:42:32.703141 13157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:32.714519 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:32.720625 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.722510 13169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.723177 13172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.723963 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:32.725147 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:32.725332 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:32.725538 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081352725500 us; error 0 us; skew 500 ppm
I20260807 05:42:32.726171 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:32.729260 10684 webserver.cc:460] Webserver started at http://127.10.111.1:45495/ using document root <none> and password file <none>
I20260807 05:42:32.729830 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:32.730029 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:32.730314 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:32.731520 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "63c152e27caf454eaf34338f97c2fbc8"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.736176 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:42:32.739826 13179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.740782 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:32.741083 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "63c152e27caf454eaf34338f97c2fbc8"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.741334 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:32.758044 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:32.759148 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:32.760700 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:32.762993 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:32.763206 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.763449 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:32.763630 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.805645 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:43419
I20260807 05:42:32.805734 13246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:43419 every 8 connection(s)
I20260807 05:42:32.810472 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:32.818451 13251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.820097 13252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.822809 13255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.825222 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:32.826407 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:32.826686 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:32.826884 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081352826867 us; error 0 us; skew 500 ppm
I20260807 05:42:32.826915 13247 heartbeater.cc:347] Connected to a master server at 127.10.111.62:35683
I20260807 05:42:32.827404 13247 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:32.827533 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:32.828218 13247 heartbeater.cc:511] Master 127.10.111.62:35683 requested a full tablet report, sending...
I20260807 05:42:32.830665 10684 webserver.cc:460] Webserver started at http://127.10.111.2:39329/ using document root <none> and password file <none>
I20260807 05:42:32.830729 13111 ts_manager.cc:194] Registered new tserver with Master: 63c152e27caf454eaf34338f97c2fbc8 (127.10.111.1:43419)
I20260807 05:42:32.831496 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:32.831784 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:32.832105 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:32.833598 13111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52074
I20260807 05:42:32.833590 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "6c13f27167e4482f866743903d24b679"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.838752 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:42:32.842573 13260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.843533 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:32.843840 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "6c13f27167e4482f866743903d24b679"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.844110 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:32.871312 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:32.872515 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:32.874133 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:32.876380 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:32.876554 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.876873 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:32.877027 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.919862 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:41249
I20260807 05:42:32.919975 13325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:41249 every 8 connection(s)
I20260807 05:42:32.924949 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:32.932660 13329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:32.933255 13330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.941133 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:32.942245 13326 heartbeater.cc:347] Connected to a master server at 127.10.111.62:35683
W20260807 05:42:32.942237 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:32.942925 13326 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:32.943603 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:32.943809 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:32.943936 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081352943927 us; error 0 us; skew 500 ppm
I20260807 05:42:32.943995 13326 heartbeater.cc:511] Master 127.10.111.62:35683 requested a full tablet report, sending...
I20260807 05:42:32.944495 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:32.946545 13111 ts_manager.cc:194] Registered new tserver with Master: 6c13f27167e4482f866743903d24b679 (127.10.111.2:41249)
I20260807 05:42:32.947655 10684 webserver.cc:460] Webserver started at http://127.10.111.3:38511/ using document root <none> and password file <none>
I20260807 05:42:32.948315 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:32.948547 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:32.948870 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:32.948889 13111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52084
I20260807 05:42:32.950407 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root/instance:
uuid: "4ef7befa4e1442099d64b1eb41ab1412"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.955027 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 05:42:32.958413 13339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.959189 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:42:32.959496 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root
uuid: "4ef7befa4e1442099d64b1eb41ab1412"
format_stamp: "Formatted at 2026-08-07 05:42:32 on dist-test-slave-qg60"
I20260807 05:42:32.959770 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:32.987368 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:32.988557 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:32.990134 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:32.992466 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:32.992642 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:32.992964 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:32.993135 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:33.036290 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.3:34821
I20260807 05:42:33.036403 13408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.3:34821 every 8 connection(s)
I20260807 05:42:33.052466 13409 heartbeater.cc:347] Connected to a master server at 127.10.111.62:35683
I20260807 05:42:33.052891 13409 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:33.053525 13409 heartbeater.cc:511] Master 127.10.111.62:35683 requested a full tablet report, sending...
I20260807 05:42:33.055325 13111 ts_manager.cc:194] Registered new tserver with Master: 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821)
I20260807 05:42:33.055682 10684 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016295535s
I20260807 05:42:33.057622 13111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52086
I20260807 05:42:33.093676 13111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260807 05:42:33.095309 13111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:42:33.156268 13289 tablet_service.cc:1467] Processing CreateTablet for tablet aa5661d94f144e5aad747aaf22f7e130 (DEFAULT_TABLE table=test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]), partition=
I20260807 05:42:33.156893 13210 tablet_service.cc:1467] Processing CreateTablet for tablet aa5661d94f144e5aad747aaf22f7e130 (DEFAULT_TABLE table=test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]), partition=
I20260807 05:42:33.157454 13289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5661d94f144e5aad747aaf22f7e130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:33.157996 13210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5661d94f144e5aad747aaf22f7e130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:33.159322 13370 tablet_service.cc:1467] Processing CreateTablet for tablet aa5661d94f144e5aad747aaf22f7e130 (DEFAULT_TABLE table=test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]), partition=
I20260807 05:42:33.160451 13370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5661d94f144e5aad747aaf22f7e130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:33.170413 13426 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Bootstrap starting.
I20260807 05:42:33.177006 13427 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Bootstrap starting.
I20260807 05:42:33.179380 13426 tablet_bootstrap.cc:654] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:33.181068 13428 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Bootstrap starting.
I20260807 05:42:33.183531 13427 tablet_bootstrap.cc:654] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:33.185248 13426 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: No bootstrap required, opened a new log
I20260807 05:42:33.185762 13426 ts_tablet_manager.cc:1397] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260807 05:42:33.188764 13428 tablet_bootstrap.cc:654] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:33.188611 13426 raft_consensus.cc:348] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.189352 13427 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: No bootstrap required, opened a new log
I20260807 05:42:33.189469 13426 raft_consensus.cc:374] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:33.189832 13427 ts_tablet_manager.cc:1397] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.001s
I20260807 05:42:33.189814 13426 raft_consensus.cc:729] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c152e27caf454eaf34338f97c2fbc8, State: Initialized, Role: FOLLOWER
I20260807 05:42:33.190786 13426 consensus_queue.cc:260] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.193033 13427 raft_consensus.cc:348] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.193817 13427 raft_consensus.cc:374] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:33.193835 13426 ts_tablet_manager.cc:1428] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 05:42:33.193912 13428 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: No bootstrap required, opened a new log
I20260807 05:42:33.194310 13427 raft_consensus.cc:729] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c13f27167e4482f866743903d24b679, State: Initialized, Role: FOLLOWER
W20260807 05:42:33.194511 13426 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 05:42:33.194519 13247 heartbeater.cc:503] Master 127.10.111.62:35683 was elected leader, sending a full tablet report...
I20260807 05:42:33.194732 13428 ts_tablet_manager.cc:1397] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260807 05:42:33.195218 13427 consensus_queue.cc:260] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.198553 13326 heartbeater.cc:503] Master 127.10.111.62:35683 was elected leader, sending a full tablet report...
I20260807 05:42:33.199043 13427 ts_tablet_manager.cc:1428] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
W20260807 05:42:33.199321 13427 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 05:42:33.197357 13428 raft_consensus.cc:348] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.200762 13428 raft_consensus.cc:374] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:33.201808 13428 raft_consensus.cc:729] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef7befa4e1442099d64b1eb41ab1412, State: Initialized, Role: FOLLOWER
I20260807 05:42:33.202513 13428 consensus_queue.cc:260] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.204926 13409 heartbeater.cc:503] Master 127.10.111.62:35683 was elected leader, sending a full tablet report...
I20260807 05:42:33.205317 13428 ts_tablet_manager.cc:1428] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260807 05:42:33.205590 13428 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
W20260807 05:42:33.290096 13410 tablet.cc:2369] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:42:33.504434 13433 raft_consensus.cc:482] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:42:33.504985 13433 raft_consensus.cc:504] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.506906 13433 leader_election.cc:288] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821), 63c152e27caf454eaf34338f97c2fbc8 (127.10.111.1:43419)
I20260807 05:42:33.528952 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5661d94f144e5aad747aaf22f7e130" candidate_uuid: "6c13f27167e4482f866743903d24b679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef7befa4e1442099d64b1eb41ab1412" is_pre_election: true
I20260807 05:42:33.529127 13220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5661d94f144e5aad747aaf22f7e130" candidate_uuid: "6c13f27167e4482f866743903d24b679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c152e27caf454eaf34338f97c2fbc8" is_pre_election: true
I20260807 05:42:33.529666 13380 raft_consensus.cc:2454] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c13f27167e4482f866743903d24b679 in term 0.
I20260807 05:42:33.529845 13220 raft_consensus.cc:2454] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c13f27167e4482f866743903d24b679 in term 0.
I20260807 05:42:33.531160 13262 leader_election.cc:302] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef7befa4e1442099d64b1eb41ab1412, 6c13f27167e4482f866743903d24b679; no voters:
I20260807 05:42:33.531919 13433 raft_consensus.cc:2781] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:42:33.532269 13433 raft_consensus.cc:482] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:42:33.532552 13433 raft_consensus.cc:3037] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:33.537191 13433 raft_consensus.cc:504] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.538642 13433 leader_election.cc:288] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [CANDIDATE]: Term 1 election: Requested vote from peers 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821), 63c152e27caf454eaf34338f97c2fbc8 (127.10.111.1:43419)
I20260807 05:42:33.539589 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5661d94f144e5aad747aaf22f7e130" candidate_uuid: "6c13f27167e4482f866743903d24b679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef7befa4e1442099d64b1eb41ab1412"
I20260807 05:42:33.539717 13220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5661d94f144e5aad747aaf22f7e130" candidate_uuid: "6c13f27167e4482f866743903d24b679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c152e27caf454eaf34338f97c2fbc8"
I20260807 05:42:33.540138 13380 raft_consensus.cc:3037] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:33.540311 13220 raft_consensus.cc:3037] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:33.545320 13220 raft_consensus.cc:2454] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c13f27167e4482f866743903d24b679 in term 1.
I20260807 05:42:33.545539 13380 raft_consensus.cc:2454] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c13f27167e4482f866743903d24b679 in term 1.
I20260807 05:42:33.546274 13261 leader_election.cc:302] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63c152e27caf454eaf34338f97c2fbc8, 6c13f27167e4482f866743903d24b679; no voters:
I20260807 05:42:33.546995 13433 raft_consensus.cc:2781] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:33.547852 13433 raft_consensus.cc:686] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 LEADER]: Becoming Leader. State: Replica: 6c13f27167e4482f866743903d24b679, State: Running, Role: LEADER
I20260807 05:42:33.548497 13433 consensus_queue.cc:237] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.555315 13111 catalog_manager.cc:5495] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c13f27167e4482f866743903d24b679 (127.10.111.2). New cstate: current_term: 1 leader_uuid: "6c13f27167e4482f866743903d24b679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } health_report { overall_health: UNKNOWN } } }
I20260807 05:42:33.657302 13370 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:38042
I20260807 05:42:33.658965 13440 tablet_replica.cc:330] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: stopping tablet replica
I20260807 05:42:33.659588 13440 raft_consensus.cc:2229] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:33.660024 13440 raft_consensus.cc:2258] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:33.662122 13440 ts_tablet_manager.cc:1905] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 05:42:33.671931 13440 ts_tablet_manager.cc:1918] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260807 05:42:33.672303 13440 log.cc:1196] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root/wals/aa5661d94f144e5aad747aaf22f7e130
I20260807 05:42:33.676606 13441 ts_tablet_manager.cc:927] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Initiating tablet copy from peer 6c13f27167e4482f866743903d24b679 (127.10.111.2:41249)
I20260807 05:42:33.678284 13441 tablet_copy_client.cc:316] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet copy: Beginning tablet copy session from remote peer at address 127.10.111.2:41249
I20260807 05:42:33.690398 13311 tablet_copy_service.cc:140] P 6c13f27167e4482f866743903d24b679: Received BeginTabletCopySession request for tablet aa5661d94f144e5aad747aaf22f7e130 from peer 4ef7befa4e1442099d64b1eb41ab1412 ({username='slave'} at 127.0.0.1:47544)
I20260807 05:42:33.690954 13311 tablet_copy_service.cc:161] P 6c13f27167e4482f866743903d24b679: Beginning new tablet copy session on tablet aa5661d94f144e5aad747aaf22f7e130 from peer 4ef7befa4e1442099d64b1eb41ab1412 at {username='slave'} at 127.0.0.1:47544: session id = 4ef7befa4e1442099d64b1eb41ab1412-aa5661d94f144e5aad747aaf22f7e130
I20260807 05:42:33.696671 13311 tablet_copy_source_session.cc:205] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:42:33.699270 13441 ts_tablet_manager.cc:1905] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260807 05:42:33.708937 13441 ts_tablet_manager.cc:1918] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260807 05:42:33.709518 13441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5661d94f144e5aad747aaf22f7e130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:33.714602 13441 tablet_copy_client.cc:791] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet copy: Starting download of 0 data blocks...
I20260807 05:42:33.715191 13441 tablet_copy_client.cc:655] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet copy: Starting download of 1 WAL segments...
I20260807 05:42:33.717000 13111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52102:
table { table_id: "980d5a72493e4e0fa588ca62a8b2edcc" table_name: "test-table" }
I20260807 05:42:33.717728 13111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52102:
table { table_id: "980d5a72493e4e0fa588ca62a8b2edcc" table_name: "test-table" }
I20260807 05:42:33.719596 13441 tablet_copy_client.cc:531] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:42:33.727228 13441 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Bootstrap starting.
I20260807 05:42:33.728825 13111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: Sending DeleteTablet for 3 replicas of tablet aa5661d94f144e5aad747aaf22f7e130
I20260807 05:42:33.731062 13289 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:42:33 UTC) from {username='slave'} at 127.0.0.1:37988
I20260807 05:42:33.731266 13370 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:42:33 UTC) from {username='slave'} at 127.0.0.1:37602
I20260807 05:42:33.732261 13210 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:42:33 UTC) from {username='slave'} at 127.0.0.1:56584
W20260807 05:42:33.734041 13092 catalog_manager.cc:4821] TS 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821): delete failed for tablet aa5661d94f144e5aad747aaf22f7e130 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa5661d94f144e5aad747aaf22f7e130 already in progress: copying tablet
I20260807 05:42:33.737223 13445 tablet_replica.cc:330] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: stopping tablet replica
I20260807 05:42:33.737372 13446 tablet_replica.cc:330] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: stopping tablet replica
I20260807 05:42:33.738092 13446 raft_consensus.cc:2229] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:33.738051 13445 raft_consensus.cc:2229] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:33.738781 13446 raft_consensus.cc:2258] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:33.739120 13445 pending_rounds.cc:62] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Trying to abort 1 pending ops.
I20260807 05:42:33.739377 13445 pending_rounds.cc:68] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315789224141516800 op_type: NO_OP noop_request { }
I20260807 05:42:33.739815 13445 raft_consensus.cc:2866] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 05:42:33.740217 13445 raft_consensus.cc:2258] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:33.741421 13446 ts_tablet_manager.cc:1905] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:42:33.742843 13445 ts_tablet_manager.cc:1905] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:42:33.745016 13441 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 05:42:33.745846 13441 tablet_bootstrap.cc:492] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Bootstrap complete.
I20260807 05:42:33.746544 13441 ts_tablet_manager.cc:1397] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.005s
I20260807 05:42:33.748566 13441 raft_consensus.cc:348] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.749421 13441 raft_consensus.cc:729] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef7befa4e1442099d64b1eb41ab1412, State: Initialized, Role: FOLLOWER
I20260807 05:42:33.750061 13441 consensus_queue.cc:260] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c13f27167e4482f866743903d24b679" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 41249 } } peers { permanent_uuid: "4ef7befa4e1442099d64b1eb41ab1412" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34821 } } peers { permanent_uuid: "63c152e27caf454eaf34338f97c2fbc8" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 43419 } }
I20260807 05:42:33.754258 13441 ts_tablet_manager.cc:1428] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
W20260807 05:42:33.754514 13441 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 05:42:33.756979 13446 ts_tablet_manager.cc:1918] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 05:42:33.757380 13446 log.cc:1196] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-0-root/wals/aa5661d94f144e5aad747aaf22f7e130
I20260807 05:42:33.758154 13446 ts_tablet_manager.cc:1939] T aa5661d94f144e5aad747aaf22f7e130 P 63c152e27caf454eaf34338f97c2fbc8: Deleting consensus metadata
I20260807 05:42:33.758718 13370 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:42:33 UTC) from {username='slave'} at 127.0.0.1:37602
I20260807 05:42:33.759644 13445 ts_tablet_manager.cc:1918] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 05:42:33.760015 13445 log.cc:1196] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-1-root/wals/aa5661d94f144e5aad747aaf22f7e130
W20260807 05:42:33.760571 13092 catalog_manager.cc:4821] TS 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821): delete failed for tablet aa5661d94f144e5aad747aaf22f7e130 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa5661d94f144e5aad747aaf22f7e130 already in progress: copying tablet
I20260807 05:42:33.760790 13445 ts_tablet_manager.cc:1939] T aa5661d94f144e5aad747aaf22f7e130 P 6c13f27167e4482f866743903d24b679: Deleting consensus metadata
I20260807 05:42:33.761809 13091 catalog_manager.cc:4841] TS 63c152e27caf454eaf34338f97c2fbc8 (127.10.111.1:43419): tablet aa5661d94f144e5aad747aaf22f7e130 (table test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]) successfully deleted
I20260807 05:42:33.762116 13095 catalog_manager.cc:4841] TS 6c13f27167e4482f866743903d24b679 (127.10.111.2:41249): tablet aa5661d94f144e5aad747aaf22f7e130 (table test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]) successfully deleted
I20260807 05:42:33.806638 13311 tablet_copy_service.cc:342] P 6c13f27167e4482f866743903d24b679: Request end of tablet copy session 4ef7befa4e1442099d64b1eb41ab1412-aa5661d94f144e5aad747aaf22f7e130 received from {username='slave'} at 127.0.0.1:47544
I20260807 05:42:33.807124 13311 tablet_copy_service.cc:433] P 6c13f27167e4482f866743903d24b679: ending tablet copy session 4ef7befa4e1442099d64b1eb41ab1412-aa5661d94f144e5aad747aaf22f7e130 on tablet aa5661d94f144e5aad747aaf22f7e130 with peer 4ef7befa4e1442099d64b1eb41ab1412
I20260807 05:42:33.810720 13370 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5661d94f144e5aad747aaf22f7e130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:42:33 UTC) from {username='slave'} at 127.0.0.1:37602
I20260807 05:42:33.811412 13440 tablet_replica.cc:330] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: stopping tablet replica
I20260807 05:42:33.812019 13440 raft_consensus.cc:2229] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:33.812397 13440 pending_rounds.cc:62] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Trying to abort 1 pending ops.
I20260807 05:42:33.812592 13440 pending_rounds.cc:68] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315789224141516800 op_type: NO_OP noop_request { }
I20260807 05:42:33.812966 13440 raft_consensus.cc:2866] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 05:42:33.813320 13440 raft_consensus.cc:2258] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:33.815274 13440 ts_tablet_manager.cc:1905] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:42:33.824798 13440 ts_tablet_manager.cc:1918] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 05:42:33.825138 13440 log.cc:1196] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786081299864053-10684-0/minicluster-data/ts-2-root/wals/aa5661d94f144e5aad747aaf22f7e130
I20260807 05:42:33.825750 13440 ts_tablet_manager.cc:1939] T aa5661d94f144e5aad747aaf22f7e130 P 4ef7befa4e1442099d64b1eb41ab1412: Deleting consensus metadata
I20260807 05:42:33.828095 13092 catalog_manager.cc:4841] TS 4ef7befa4e1442099d64b1eb41ab1412 (127.10.111.3:34821): tablet aa5661d94f144e5aad747aaf22f7e130 (table test-table [id=980d5a72493e4e0fa588ca62a8b2edcc]) successfully deleted
I20260807 05:42:33.869429 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:33.890681 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:33.908538 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:33.918706 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:42:33.938560 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:33.956097 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:42:33.965866 10684 tablet_server.cc:178] TabletServer@127.10.111.3:0 shutting down...
I20260807 05:42:33.983758 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:34.001073 10684 tablet_server.cc:194] TabletServer@127.10.111.3:0 shutdown complete.
I20260807 05:42:34.011168 10684 master.cc:537] Master@127.10.111.62:35683 shutting down...
I20260807 05:42:34.026855 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:34.027421 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:34.027752 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P a29dfa48c1564f2f8a800be45c3ccfed: stopping tablet replica
I20260807 05:42:34.047420 10684 master.cc:552] Master@127.10.111.62:35683 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1571 ms)
[----------] 4 tests from TsTabletManagerITest (43045 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260807 05:42:34.072206 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:42351
I20260807 05:42:34.073316 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:34.078660 13454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:34.079737 13455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.082273 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:34.082481 13457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.083321 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:34.083523 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:34.083652 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081354083642 us; error 0 us; skew 500 ppm
I20260807 05:42:34.084112 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:34.086472 10684 webserver.cc:460] Webserver started at http://127.10.111.62:43095/ using document root <none> and password file <none>
I20260807 05:42:34.086933 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:34.087093 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:34.087301 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:34.088433 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "1243c9998fbf4cec91c01eeac2780209"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:34.093856 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 05:42:34.097350 13464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:34.098184 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 05:42:34.098474 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "1243c9998fbf4cec91c01eeac2780209"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:34.098769 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:34.114408 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:34.115434 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:34.152650 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:42351
I20260807 05:42:34.152804 13517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:42351 every 8 connection(s)
I20260807 05:42:34.156523 13518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:34.166611 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209: Bootstrap starting.
I20260807 05:42:34.171268 13518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:34.175544 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209: No bootstrap required, opened a new log
I20260807 05:42:34.177486 13518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER }
I20260807 05:42:34.177881 13518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:34.178064 13518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1243c9998fbf4cec91c01eeac2780209, State: Initialized, Role: FOLLOWER
I20260807 05:42:34.178701 13518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER }
I20260807 05:42:34.179121 13518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:34.179303 13518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:34.179510 13518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:34.183817 13518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER }
I20260807 05:42:34.184288 13518 leader_election.cc:302] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1243c9998fbf4cec91c01eeac2780209; no voters:
I20260807 05:42:34.185415 13518 leader_election.cc:288] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:34.185762 13521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:34.187119 13521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 1 LEADER]: Becoming Leader. State: Replica: 1243c9998fbf4cec91c01eeac2780209, State: Running, Role: LEADER
I20260807 05:42:34.187786 13521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER }
I20260807 05:42:34.188385 13518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:34.192301 13522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1243c9998fbf4cec91c01eeac2780209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER } }
I20260807 05:42:34.193125 13522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:34.193306 13523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1243c9998fbf4cec91c01eeac2780209. Latest consensus state: current_term: 1 leader_uuid: "1243c9998fbf4cec91c01eeac2780209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1243c9998fbf4cec91c01eeac2780209" member_type: VOTER } }
I20260807 05:42:34.194032 13523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:34.198791 13530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:34.206581 13530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:34.207870 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:34.216218 13530 catalog_manager.cc:1347] Generated new cluster ID: 984d099073504502aab3445d873b880c
I20260807 05:42:34.216542 13530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:34.230866 13530 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:34.232872 13530 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:34.246573 13530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209: Generated new TSK 0
I20260807 05:42:34.247349 13530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:34.275399 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:34.281569 13541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:34.283200 13543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.285466 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:34.285425 13546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.286775 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:34.287017 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:34.287189 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081354287172 us; error 0 us; skew 500 ppm
I20260807 05:42:34.287719 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:34.290416 10684 webserver.cc:460] Webserver started at http://127.10.111.1:45029/ using document root <none> and password file <none>
I20260807 05:42:34.290936 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:34.291153 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:34.291414 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:34.292562 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "beec49e77bfc4bc3bb71288e6b158898"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:34.297380 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 05:42:34.301702 13551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:34.302538 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:42:34.302839 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "beec49e77bfc4bc3bb71288e6b158898"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:34.303148 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:34.332216 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:34.333416 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:34.338986 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:34.339355 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:34.339731 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:34.340006 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:34.394524 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:39287
I20260807 05:42:34.394634 13623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:39287 every 8 connection(s)
I20260807 05:42:34.399303 13624 heartbeater.cc:347] Connected to a master server at 127.10.111.62:42351
I20260807 05:42:34.399712 13624 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:34.400559 13624 heartbeater.cc:511] Master 127.10.111.62:42351 requested a full tablet report, sending...
I20260807 05:42:34.402465 13481 ts_manager.cc:194] Registered new tserver with Master: beec49e77bfc4bc3bb71288e6b158898 (127.10.111.1:39287)
I20260807 05:42:34.402861 10684 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004938777s
I20260807 05:42:34.405395 13481 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49612
I20260807 05:42:34.440398 13588 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 05:42:34.441896 13588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:34.454195 13635 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Bootstrap starting.
I20260807 05:42:34.458737 13635 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:34.462774 13635 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: No bootstrap required, opened a new log
I20260807 05:42:34.463155 13635 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 05:42:34.465015 13635 raft_consensus.cc:348] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.465440 13635 raft_consensus.cc:374] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:34.465688 13635 raft_consensus.cc:729] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beec49e77bfc4bc3bb71288e6b158898, State: Initialized, Role: FOLLOWER
I20260807 05:42:34.466233 13635 consensus_queue.cc:260] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.466707 13635 raft_consensus.cc:388] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:34.466945 13635 raft_consensus.cc:482] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:34.467203 13635 raft_consensus.cc:3037] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:34.472205 13635 raft_consensus.cc:504] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.472959 13635 leader_election.cc:302] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beec49e77bfc4bc3bb71288e6b158898; no voters:
I20260807 05:42:34.474262 13635 leader_election.cc:288] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:34.474665 13637 raft_consensus.cc:2781] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:34.476161 13637 raft_consensus.cc:686] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 LEADER]: Becoming Leader. State: Replica: beec49e77bfc4bc3bb71288e6b158898, State: Running, Role: LEADER
I20260807 05:42:34.476456 13624 heartbeater.cc:503] Master 127.10.111.62:42351 was elected leader, sending a full tablet report...
I20260807 05:42:34.477005 13635 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 05:42:34.477052 13637 consensus_queue.cc:237] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.483946 13638 tablet_replica.cc:439] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beec49e77bfc4bc3bb71288e6b158898" committed_config { opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } } }
I20260807 05:42:34.484522 13638 tablet_replica.cc:442] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: This TxnStatusTablet replica's current role is: LEADER
W20260807 05:42:34.485343 13481 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:34.485505 13639 tablet_replica.cc:439] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: TxnStatusTablet state changed. Reason: New leader beec49e77bfc4bc3bb71288e6b158898. Latest consensus state: current_term: 1 leader_uuid: "beec49e77bfc4bc3bb71288e6b158898" committed_config { opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } } }
I20260807 05:42:34.486085 13642 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:42:34.486117 13639 tablet_replica.cc:442] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:34.486918 13642 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 05:42:34.490322 13481 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:34.522971 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:34.554584 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:34.555799 10684 tablet_replica.cc:330] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: stopping tablet replica
I20260807 05:42:34.556386 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:34.556936 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:34.579720 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:34.595520 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:34.602438 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:34.604809 13647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:34.608316 13649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.609282 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:34.610394 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:34.610615 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:34.610785 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081354610770 us; error 0 us; skew 500 ppm
I20260807 05:42:34.611321 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:34.614354 10684 webserver.cc:460] Webserver started at http://127.10.111.1:45029/ using document root <none> and password file <none>
I20260807 05:42:34.614866 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:34.615062 10684 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 05:42:34.619045 10684 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:42:34.622083 13655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:34.622965 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:34.623311 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "beec49e77bfc4bc3bb71288e6b158898"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:34.623728 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:34.650357 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:34.651726 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:34.661165 13664 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 05:42:34.669831 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:42:34.670130 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.002s sys 0.000s
I20260807 05:42:34.670491 10684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:42:34.675912 10684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:42:34.676190 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.006s user 0.003s sys 0.000s
I20260807 05:42:34.676365 13664 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Bootstrap starting.
I20260807 05:42:34.730441 13664 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:34.731572 13664 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Bootstrap complete.
I20260807 05:42:34.732355 13664 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Time spent bootstrapping tablet: real 0.056s user 0.043s sys 0.004s
I20260807 05:42:34.735365 13664 raft_consensus.cc:348] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.736019 13664 raft_consensus.cc:729] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: beec49e77bfc4bc3bb71288e6b158898, State: Initialized, Role: FOLLOWER
I20260807 05:42:34.737697 13664 consensus_queue.cc:260] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.738444 13664 raft_consensus.cc:388] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:34.738782 13664 raft_consensus.cc:482] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:34.739161 13664 raft_consensus.cc:3037] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:42:34.748315 13664 raft_consensus.cc:504] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.749203 13664 leader_election.cc:302] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beec49e77bfc4bc3bb71288e6b158898; no voters:
I20260807 05:42:34.753353 13664 leader_election.cc:288] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:42:34.753898 13681 raft_consensus.cc:2781] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:42:34.764146 13664 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898: Time spent starting tablet: real 0.031s user 0.018s sys 0.012s
I20260807 05:42:34.765781 13681 raft_consensus.cc:686] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 2 LEADER]: Becoming Leader. State: Replica: beec49e77bfc4bc3bb71288e6b158898, State: Running, Role: LEADER
I20260807 05:42:34.766971 13681 consensus_queue.cc:237] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } }
I20260807 05:42:34.777652 13682 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "beec49e77bfc4bc3bb71288e6b158898" committed_config { opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } } }
I20260807 05:42:34.777652 13685 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader beec49e77bfc4bc3bb71288e6b158898. Latest consensus state: current_term: 2 leader_uuid: "beec49e77bfc4bc3bb71288e6b158898" committed_config { opid_index: -1 peers { permanent_uuid: "beec49e77bfc4bc3bb71288e6b158898" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 39287 } } }
I20260807 05:42:34.778401 13682 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:34.778415 13685 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:34.801710 13700 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:42:34.802213 13700 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:42:34.835873 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:39287
I20260807 05:42:34.835954 13738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:39287 every 8 connection(s)
I20260807 05:42:34.842221 13739 heartbeater.cc:347] Connected to a master server at 127.10.111.62:42351
I20260807 05:42:34.842631 13739 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:34.843394 13739 heartbeater.cc:511] Master 127.10.111.62:42351 requested a full tablet report, sending...
I20260807 05:42:34.846467 13481 ts_manager.cc:194] Re-registered known tserver with Master: beec49e77bfc4bc3bb71288e6b158898 (127.10.111.1:39287)
W20260807 05:42:34.847038 13481 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:34.847918 10684 tablet_server.cc:178] TabletServer@127.10.111.1:39287 shutting down...
I20260807 05:42:34.848809 13481 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49638
I20260807 05:42:34.853045 13739 heartbeater.cc:503] Master 127.10.111.62:42351 was elected leader, sending a full tablet report...
W20260807 05:42:34.856535 13481 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:34.873508 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:34.874410 10684 tablet_replica.cc:330] stopping tablet replica
I20260807 05:42:34.874926 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:42:34.875373 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P beec49e77bfc4bc3bb71288e6b158898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:34.896471 10684 tablet_server.cc:194] TabletServer@127.10.111.1:39287 shutdown complete.
I20260807 05:42:34.907775 10684 master.cc:537] Master@127.10.111.62:42351 shutting down...
I20260807 05:42:34.926672 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:34.927312 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:34.927688 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1243c9998fbf4cec91c01eeac2780209: stopping tablet replica
I20260807 05:42:34.946916 10684 master.cc:552] Master@127.10.111.62:42351 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (905 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260807 05:42:34.979873 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:43105
I20260807 05:42:34.981285 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:34.987274 13745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:34.988195 13746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.991178 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:34.991804 13748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:34.992820 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:34.993058 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:34.993229 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081354993215 us; error 0 us; skew 500 ppm
I20260807 05:42:34.993779 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:34.996824 10684 webserver.cc:460] Webserver started at http://127.10.111.62:39959/ using document root <none> and password file <none>
I20260807 05:42:34.997331 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:34.997538 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:34.997817 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:34.998921 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "22674d549af04e0d904439d00dce276b"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:35.003942 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 05:42:35.007929 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.008944 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:35.009261 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "22674d549af04e0d904439d00dce276b"
format_stamp: "Formatted at 2026-08-07 05:42:34 on dist-test-slave-qg60"
I20260807 05:42:35.009534 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:35.028472 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:35.029569 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:35.082101 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:43105
I20260807 05:42:35.082214 13807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:43105 every 8 connection(s)
I20260807 05:42:35.086521 13809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:35.101703 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b: Bootstrap starting.
I20260807 05:42:35.107666 13809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:35.112571 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b: No bootstrap required, opened a new log
I20260807 05:42:35.114511 13809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER }
I20260807 05:42:35.114944 13809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:35.115183 13809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22674d549af04e0d904439d00dce276b, State: Initialized, Role: FOLLOWER
I20260807 05:42:35.115805 13809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER }
I20260807 05:42:35.116298 13809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:35.116542 13809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:35.116824 13809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:35.121408 13809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER }
I20260807 05:42:35.121917 13809 leader_election.cc:302] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22674d549af04e0d904439d00dce276b; no voters:
I20260807 05:42:35.123334 13809 leader_election.cc:288] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:35.123716 13812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:35.125799 13812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 1 LEADER]: Becoming Leader. State: Replica: 22674d549af04e0d904439d00dce276b, State: Running, Role: LEADER
I20260807 05:42:35.126533 13812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER }
I20260807 05:42:35.127391 13809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:35.129731 13813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22674d549af04e0d904439d00dce276b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER } }
I20260807 05:42:35.130285 13813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:35.131598 13814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22674d549af04e0d904439d00dce276b. Latest consensus state: current_term: 1 leader_uuid: "22674d549af04e0d904439d00dce276b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22674d549af04e0d904439d00dce276b" member_type: VOTER } }
I20260807 05:42:35.132313 13814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:35.143651 13818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:35.150493 13818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:35.153273 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:35.160354 13818 catalog_manager.cc:1347] Generated new cluster ID: 6d89db2d52544f90a6c7d135888bd5e5
I20260807 05:42:35.160642 13818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:35.180989 13818 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:35.183378 13818 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:35.204704 13818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b: Generated new TSK 0
I20260807 05:42:35.205611 13818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:35.219666 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:35.226584 13833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.227651 13834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.231843 13836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:35.232159 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:35.233302 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:35.233486 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:35.233676 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081355233645 us; error 0 us; skew 500 ppm
I20260807 05:42:35.234197 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:35.237154 10684 webserver.cc:460] Webserver started at http://127.10.111.1:35607/ using document root <none> and password file <none>
I20260807 05:42:35.237634 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:35.237838 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:35.238121 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:35.239244 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "3ccf0ee6ff864cadba1875c942748604"
format_stamp: "Formatted at 2026-08-07 05:42:35 on dist-test-slave-qg60"
I20260807 05:42:35.244079 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:42:35.247923 13841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.248911 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:35.249388 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "3ccf0ee6ff864cadba1875c942748604"
format_stamp: "Formatted at 2026-08-07 05:42:35 on dist-test-slave-qg60"
I20260807 05:42:35.249945 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:35.270123 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:35.271466 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:35.288798 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:35.289139 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 05:42:35.289445 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:35.289654 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.363565 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:38901
I20260807 05:42:35.363673 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:38901 every 8 connection(s)
I20260807 05:42:35.369545 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:42:35.369768 13915 heartbeater.cc:347] Connected to a master server at 127.10.111.62:43105
I20260807 05:42:35.370242 13915 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:35.371017 13915 heartbeater.cc:511] Master 127.10.111.62:43105 requested a full tablet report, sending...
I20260807 05:42:35.373121 13773 ts_manager.cc:194] Registered new tserver with Master: 3ccf0ee6ff864cadba1875c942748604 (127.10.111.1:38901)
I20260807 05:42:35.376000 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41410
W20260807 05:42:35.377102 13917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.380417 13918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.384163 13920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:35.384642 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:35.385517 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:35.385767 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:35.385946 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081355385920 us; error 0 us; skew 500 ppm
I20260807 05:42:35.386484 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:35.389278 10684 webserver.cc:460] Webserver started at http://127.10.111.2:41979/ using document root <none> and password file <none>
I20260807 05:42:35.389771 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:35.389972 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:35.390240 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:35.391345 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "71c20143a0cd454b883ddb1945061bea"
format_stamp: "Formatted at 2026-08-07 05:42:35 on dist-test-slave-qg60"
I20260807 05:42:35.396248 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 05:42:35.400178 13925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.401327 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:35.401644 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "71c20143a0cd454b883ddb1945061bea"
format_stamp: "Formatted at 2026-08-07 05:42:35 on dist-test-slave-qg60"
I20260807 05:42:35.401911 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:35.421778 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:35.422961 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:35.429320 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:35.429632 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.429966 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:35.430145 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.498540 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:41995
I20260807 05:42:35.498667 14001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:41995 every 8 connection(s)
I20260807 05:42:35.504091 14002 heartbeater.cc:347] Connected to a master server at 127.10.111.62:43105
I20260807 05:42:35.504530 14002 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:35.505354 14002 heartbeater.cc:511] Master 127.10.111.62:43105 requested a full tablet report, sending...
I20260807 05:42:35.507547 13773 ts_manager.cc:194] Registered new tserver with Master: 71c20143a0cd454b883ddb1945061bea (127.10.111.2:41995)
I20260807 05:42:35.508589 10684 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.00645964s
I20260807 05:42:35.509690 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41416
I20260807 05:42:35.549201 13878 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 05:42:35.550385 13878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:35.561386 14014 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Bootstrap starting.
I20260807 05:42:35.565635 14014 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:35.569965 14014 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: No bootstrap required, opened a new log
I20260807 05:42:35.570386 14014 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 05:42:35.572268 14014 raft_consensus.cc:348] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.572769 14014 raft_consensus.cc:374] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:35.573014 14014 raft_consensus.cc:729] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ccf0ee6ff864cadba1875c942748604, State: Initialized, Role: FOLLOWER
I20260807 05:42:35.573550 14014 consensus_queue.cc:260] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.574055 14014 raft_consensus.cc:388] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:35.574303 14014 raft_consensus.cc:482] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:35.574559 14014 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:35.579560 14014 raft_consensus.cc:504] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.580127 14014 leader_election.cc:302] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ccf0ee6ff864cadba1875c942748604; no voters:
I20260807 05:42:35.581561 14014 leader_election.cc:288] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:35.581959 14016 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:35.583623 13915 heartbeater.cc:503] Master 127.10.111.62:43105 was elected leader, sending a full tablet report...
I20260807 05:42:35.584357 14014 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260807 05:42:35.585000 14016 raft_consensus.cc:686] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 LEADER]: Becoming Leader. State: Replica: 3ccf0ee6ff864cadba1875c942748604, State: Running, Role: LEADER
I20260807 05:42:35.585675 14016 consensus_queue.cc:237] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.588999 14017 tablet_replica.cc:439] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ccf0ee6ff864cadba1875c942748604" committed_config { opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } } }
I20260807 05:42:35.589564 14017 tablet_replica.cc:442] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: This TxnStatusTablet replica's current role is: LEADER
W20260807 05:42:35.591486 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:35.591466 14019 tablet_replica.cc:439] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: TxnStatusTablet state changed. Reason: New leader 3ccf0ee6ff864cadba1875c942748604. Latest consensus state: current_term: 1 leader_uuid: "3ccf0ee6ff864cadba1875c942748604" committed_config { opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } } }
I20260807 05:42:35.593499 14019 tablet_replica.cc:442] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:35.599687 14021 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:42:35.600112 14021 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 05:42:35.601819 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:35.661877 14026 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Initiating tablet copy from peer 3ccf0ee6ff864cadba1875c942748604 (127.10.111.1:38901)
I20260807 05:42:35.663491 14026 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: tablet copy: Beginning tablet copy session from remote peer at address 127.10.111.1:38901
I20260807 05:42:35.675144 13898 tablet_copy_service.cc:140] P 3ccf0ee6ff864cadba1875c942748604: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer 71c20143a0cd454b883ddb1945061bea ({username='slave'} at 127.0.0.1:40892)
I20260807 05:42:35.675673 13898 tablet_copy_service.cc:161] P 3ccf0ee6ff864cadba1875c942748604: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer 71c20143a0cd454b883ddb1945061bea at {username='slave'} at 127.0.0.1:40892: session id = 71c20143a0cd454b883ddb1945061bea-11111111111111111111111111111111
I20260807 05:42:35.680860 13898 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:42:35.683517 14026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:35.693596 14026 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: tablet copy: Starting download of 0 data blocks...
I20260807 05:42:35.694105 14026 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: tablet copy: Starting download of 1 WAL segments...
I20260807 05:42:35.697716 14026 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:42:35.703904 14026 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Bootstrap starting.
I20260807 05:42:35.725412 14026 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:35.726190 14026 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Bootstrap complete.
I20260807 05:42:35.726796 14026 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.004s
I20260807 05:42:35.729166 14026 raft_consensus.cc:348] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.729601 14026 raft_consensus.cc:729] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 71c20143a0cd454b883ddb1945061bea, State: Initialized, Role: NON_PARTICIPANT
I20260807 05:42:35.730124 14026 consensus_queue.cc:260] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.732156 14029 tablet_replica.cc:439] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } } }
I20260807 05:42:35.732426 14002 heartbeater.cc:503] Master 127.10.111.62:43105 was elected leader, sending a full tablet report...
I20260807 05:42:35.732764 14029 tablet_replica.cc:442] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260807 05:42:35.732905 14026 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260807 05:42:35.734649 13898 tablet_copy_service.cc:342] P 3ccf0ee6ff864cadba1875c942748604: Request end of tablet copy session 71c20143a0cd454b883ddb1945061bea-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:40892
I20260807 05:42:35.735016 13898 tablet_copy_service.cc:433] P 3ccf0ee6ff864cadba1875c942748604: ending tablet copy session 71c20143a0cd454b883ddb1945061bea-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer 71c20143a0cd454b883ddb1945061bea
W20260807 05:42:35.735483 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260807 05:42:35.739660 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:35.758668 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:35.787740 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:35.788785 10684 tablet_replica.cc:330] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: stopping tablet replica
I20260807 05:42:35.789448 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:35.789986 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:35.814096 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:35.829159 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:35.836198 14031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.838881 14032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:35.840476 14034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:35.840685 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:35.841871 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:35.842108 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:35.842288 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081355842267 us; error 0 us; skew 500 ppm
I20260807 05:42:35.842801 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:35.845953 10684 webserver.cc:460] Webserver started at http://127.10.111.1:35607/ using document root <none> and password file <none>
I20260807 05:42:35.846550 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:35.846745 10684 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 05:42:35.851161 10684 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 05:42:35.854241 14039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:35.855151 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:42:35.855468 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "3ccf0ee6ff864cadba1875c942748604"
format_stamp: "Formatted at 2026-08-07 05:42:35 on dist-test-slave-qg60"
I20260807 05:42:35.855757 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:35.866163 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:35.867129 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:35.874920 14049 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 05:42:35.883720 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:42:35.884131 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.002s sys 0.000s
I20260807 05:42:35.884475 10684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:42:35.890921 10684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:42:35.891250 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.005s sys 0.000s
I20260807 05:42:35.891302 14049 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Bootstrap starting.
I20260807 05:42:35.923511 14049 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:35.924536 14049 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Bootstrap complete.
I20260807 05:42:35.925884 14049 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Time spent bootstrapping tablet: real 0.035s user 0.028s sys 0.004s
I20260807 05:42:35.928193 14049 raft_consensus.cc:348] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.928773 14049 raft_consensus.cc:729] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ccf0ee6ff864cadba1875c942748604, State: Initialized, Role: FOLLOWER
I20260807 05:42:35.929656 14049 consensus_queue.cc:260] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.930272 14049 raft_consensus.cc:388] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:35.930572 14049 raft_consensus.cc:482] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:35.930907 14049 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:42:35.936090 14049 raft_consensus.cc:504] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.936678 14049 leader_election.cc:302] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ccf0ee6ff864cadba1875c942748604; no voters:
I20260807 05:42:35.942117 14049 leader_election.cc:288] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:42:35.942840 14064 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:42:35.955505 14049 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604: Time spent starting tablet: real 0.029s user 0.012s sys 0.015s
I20260807 05:42:35.956905 14064 raft_consensus.cc:686] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 2 LEADER]: Becoming Leader. State: Replica: 3ccf0ee6ff864cadba1875c942748604, State: Running, Role: LEADER
I20260807 05:42:35.957952 14064 consensus_queue.cc:237] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } }
I20260807 05:42:35.962142 14067 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 3ccf0ee6ff864cadba1875c942748604. Latest consensus state: current_term: 2 leader_uuid: "3ccf0ee6ff864cadba1875c942748604" committed_config { opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } } }
I20260807 05:42:35.962121 14066 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3ccf0ee6ff864cadba1875c942748604" committed_config { opid_index: -1 peers { permanent_uuid: "3ccf0ee6ff864cadba1875c942748604" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 38901 } } }
I20260807 05:42:35.962980 14067 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:35.963058 14066 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:35.970112 14073 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:42:35.970685 14073 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:42:36.018190 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:38901
I20260807 05:42:36.018302 14124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:38901 every 8 connection(s)
I20260807 05:42:36.023700 14125 heartbeater.cc:347] Connected to a master server at 127.10.111.62:43105
I20260807 05:42:36.024214 14125 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:36.025103 14125 heartbeater.cc:511] Master 127.10.111.62:43105 requested a full tablet report, sending...
I20260807 05:42:36.029003 13773 ts_manager.cc:194] Re-registered known tserver with Master: 3ccf0ee6ff864cadba1875c942748604 (127.10.111.1:38901)
I20260807 05:42:36.029268 10684 tablet_server.cc:178] TabletServer@127.10.111.1:38901 shutting down...
W20260807 05:42:36.029529 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:36.033468 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41446
I20260807 05:42:36.036553 14125 heartbeater.cc:503] Master 127.10.111.62:43105 was elected leader, sending a full tablet report...
W20260807 05:42:36.039495 13773 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:36.056699 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:36.057735 10684 tablet_replica.cc:330] stopping tablet replica
I20260807 05:42:36.058357 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:42:36.058815 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 3ccf0ee6ff864cadba1875c942748604 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:36.080803 10684 tablet_server.cc:194] TabletServer@127.10.111.1:38901 shutdown complete.
I20260807 05:42:36.092561 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:42:36.111930 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:36.112856 10684 tablet_replica.cc:330] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea: stopping tablet replica
I20260807 05:42:36.113497 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 05:42:36.113971 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 71c20143a0cd454b883ddb1945061bea [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 05:42:36.135059 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:42:36.146880 10684 master.cc:537] Master@127.10.111.62:43105 shutting down...
I20260807 05:42:36.170917 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:36.171607 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:36.172020 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 22674d549af04e0d904439d00dce276b: stopping tablet replica
I20260807 05:42:36.192233 10684 master.cc:552] Master@127.10.111.62:43105 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1245 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260807 05:42:36.225871 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:41529
I20260807 05:42:36.227366 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:36.233603 14132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:36.233960 14134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:36.237998 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:36.238119 14136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:36.239166 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:36.239427 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:36.239602 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081356239588 us; error 0 us; skew 500 ppm
I20260807 05:42:36.240319 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:36.243408 10684 webserver.cc:460] Webserver started at http://127.10.111.62:46549/ using document root <none> and password file <none>
I20260807 05:42:36.244045 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:36.244267 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:36.244585 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:36.246263 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "4738d55714de47569653aa1d0283cf88"
format_stamp: "Formatted at 2026-08-07 05:42:36 on dist-test-slave-qg60"
I20260807 05:42:36.252893 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 05:42:36.257516 14142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:36.258479 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:42:36.258822 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "4738d55714de47569653aa1d0283cf88"
format_stamp: "Formatted at 2026-08-07 05:42:36 on dist-test-slave-qg60"
I20260807 05:42:36.259191 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:36.306895 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:36.308249 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:36.357091 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:41529
I20260807 05:42:36.357290 14197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:41529 every 8 connection(s)
I20260807 05:42:36.361660 14198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:36.372406 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88: Bootstrap starting.
I20260807 05:42:36.376572 14198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:36.380934 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88: No bootstrap required, opened a new log
I20260807 05:42:36.382791 14198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER }
I20260807 05:42:36.383162 14198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:36.383412 14198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4738d55714de47569653aa1d0283cf88, State: Initialized, Role: FOLLOWER
I20260807 05:42:36.384025 14198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER }
I20260807 05:42:36.384459 14198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:36.384677 14198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:36.384994 14198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:36.389263 14198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER }
I20260807 05:42:36.389775 14198 leader_election.cc:302] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4738d55714de47569653aa1d0283cf88; no voters:
I20260807 05:42:36.391058 14198 leader_election.cc:288] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:36.391396 14201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:36.393164 14201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 1 LEADER]: Becoming Leader. State: Replica: 4738d55714de47569653aa1d0283cf88, State: Running, Role: LEADER
I20260807 05:42:36.393873 14201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER }
I20260807 05:42:36.394289 14198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:36.397006 14203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4738d55714de47569653aa1d0283cf88. Latest consensus state: current_term: 1 leader_uuid: "4738d55714de47569653aa1d0283cf88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER } }
I20260807 05:42:36.396942 14202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4738d55714de47569653aa1d0283cf88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4738d55714de47569653aa1d0283cf88" member_type: VOTER } }
I20260807 05:42:36.397697 14203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:36.397787 14202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:36.414443 14206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:36.421393 14206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:36.422657 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:36.431810 14206 catalog_manager.cc:1347] Generated new cluster ID: 82c51ce0a8df4414af12721197aaffe0
I20260807 05:42:36.432149 14206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:36.457507 14206 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:36.459375 14206 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:36.475415 14206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88: Generated new TSK 0
I20260807 05:42:36.476240 14206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:36.489467 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:36.495967 14222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:36.498987 14223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:36.500133 14225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:36.503159 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:36.504073 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:36.504300 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:36.504505 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081356504459 us; error 0 us; skew 500 ppm
I20260807 05:42:36.505096 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:36.508098 10684 webserver.cc:460] Webserver started at http://127.10.111.1:41247/ using document root <none> and password file <none>
I20260807 05:42:36.508584 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:36.508811 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:36.509188 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:36.510344 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "73e3e469c0aa45e8baf32a20b42b7f8c"
format_stamp: "Formatted at 2026-08-07 05:42:36 on dist-test-slave-qg60"
I20260807 05:42:36.515348 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 05:42:36.519392 14230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:36.520311 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:42:36.520666 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "73e3e469c0aa45e8baf32a20b42b7f8c"
format_stamp: "Formatted at 2026-08-07 05:42:36 on dist-test-slave-qg60"
I20260807 05:42:36.521077 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:36.543466 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:36.544927 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:36.557839 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:36.558173 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:36.558465 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:36.558655 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:36.623927 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:36813
I20260807 05:42:36.624053 14309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:36813 every 8 connection(s)
I20260807 05:42:36.630398 14310 heartbeater.cc:347] Connected to a master server at 127.10.111.62:41529
I20260807 05:42:36.630817 14310 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:36.631640 14310 heartbeater.cc:511] Master 127.10.111.62:41529 requested a full tablet report, sending...
I20260807 05:42:36.633996 14159 ts_manager.cc:194] Registered new tserver with Master: 73e3e469c0aa45e8baf32a20b42b7f8c (127.10.111.1:36813)
I20260807 05:42:36.634529 10684 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.001455754s
I20260807 05:42:36.636674 14159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50068
I20260807 05:42:36.675558 14268 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 05:42:36.676914 14268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:36.688005 14320 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: Bootstrap starting.
I20260807 05:42:36.692312 14320 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:36.696646 14320 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: No bootstrap required, opened a new log
I20260807 05:42:36.697110 14320 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260807 05:42:36.698889 14320 raft_consensus.cc:348] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } }
I20260807 05:42:36.699328 14320 raft_consensus.cc:374] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:36.699553 14320 raft_consensus.cc:729] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e3e469c0aa45e8baf32a20b42b7f8c, State: Initialized, Role: FOLLOWER
I20260807 05:42:36.700115 14320 consensus_queue.cc:260] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } }
I20260807 05:42:36.700591 14320 raft_consensus.cc:388] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:36.700896 14320 raft_consensus.cc:482] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:36.701159 14320 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:36.705698 14320 raft_consensus.cc:504] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } }
I20260807 05:42:36.706224 14320 leader_election.cc:302] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73e3e469c0aa45e8baf32a20b42b7f8c; no voters:
I20260807 05:42:36.707661 14320 leader_election.cc:288] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:36.708019 14322 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:36.709790 14310 heartbeater.cc:503] Master 127.10.111.62:41529 was elected leader, sending a full tablet report...
I20260807 05:42:36.710423 14320 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260807 05:42:36.711018 14322 raft_consensus.cc:686] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 1 LEADER]: Becoming Leader. State: Replica: 73e3e469c0aa45e8baf32a20b42b7f8c, State: Running, Role: LEADER
I20260807 05:42:36.711696 14322 consensus_queue.cc:237] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } }
I20260807 05:42:36.714813 14323 tablet_replica.cc:439] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" committed_config { opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } } }
I20260807 05:42:36.715372 14323 tablet_replica.cc:442] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: This TxnStatusTablet replica's current role is: LEADER
W20260807 05:42:36.717219 14159 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:36.717262 14324 tablet_replica.cc:439] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: TxnStatusTablet state changed. Reason: New leader 73e3e469c0aa45e8baf32a20b42b7f8c. Latest consensus state: current_term: 1 leader_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" committed_config { opid_index: -1 peers { permanent_uuid: "73e3e469c0aa45e8baf32a20b42b7f8c" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 36813 } } }
I20260807 05:42:36.719141 14324 tablet_replica.cc:442] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:36.721623 14326 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:42:36.722046 14326 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 05:42:36.723843 14159 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260807 05:42:36.779182 14266 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 05:42:36.779495 14266 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 05:42:36.779808 14266 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260807 05:42:36.788769 14266 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 05:42:36.831970 14266 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 05:42:36.832336 14266 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260807 05:42:36.840327 14266 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260807 05:42:36.855762 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.858286 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.860883 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.863072 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.865092 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.866950 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:46880
W20260807 05:42:36.879716 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
W20260807 05:42:36.882162 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
W20260807 05:42:36.884424 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
W20260807 05:42:36.886613 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
W20260807 05:42:36.888445 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
W20260807 05:42:36.890344 14265 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:46886
I20260807 05:42:36.895478 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:36.919215 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:36.920418 10684 tablet_replica.cc:330] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c: stopping tablet replica
I20260807 05:42:36.921056 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:36.921687 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 73e3e469c0aa45e8baf32a20b42b7f8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:36.939788 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:36.958525 10684 master.cc:537] Master@127.10.111.62:41529 shutting down...
I20260807 05:42:36.986667 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:36.987215 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:36.987556 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4738d55714de47569653aa1d0283cf88: stopping tablet replica
I20260807 05:42:37.007035 10684 master.cc:552] Master@127.10.111.62:41529 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (812 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260807 05:42:37.037696 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:33985
I20260807 05:42:37.038807 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:37.044634 14335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.046320 14336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.049275 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:37.049561 14338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.050565 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:37.050748 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:37.050938 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081357050924 us; error 0 us; skew 500 ppm
I20260807 05:42:37.051786 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:37.054935 10684 webserver.cc:460] Webserver started at http://127.10.111.62:34591/ using document root <none> and password file <none>
I20260807 05:42:37.055433 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:37.055646 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:37.055923 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:37.057220 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "29138508bd154071a1b3b5befcc2859c"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.062027 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:42:37.065665 14345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.066576 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 05:42:37.066862 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "29138508bd154071a1b3b5befcc2859c"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.067153 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:37.078919 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:37.079905 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:37.117400 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:33985
I20260807 05:42:37.117532 14399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:33985 every 8 connection(s)
I20260807 05:42:37.121279 14400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:37.131485 14400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c: Bootstrap starting.
I20260807 05:42:37.135797 14400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:37.139698 14400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c: No bootstrap required, opened a new log
I20260807 05:42:37.141611 14400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER }
I20260807 05:42:37.141978 14400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:37.142158 14400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29138508bd154071a1b3b5befcc2859c, State: Initialized, Role: FOLLOWER
I20260807 05:42:37.142621 14400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER }
I20260807 05:42:37.143031 14400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:37.143239 14400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:37.143453 14400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:37.147773 14400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER }
I20260807 05:42:37.148231 14400 leader_election.cc:302] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29138508bd154071a1b3b5befcc2859c; no voters:
I20260807 05:42:37.149439 14400 leader_election.cc:288] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:37.149766 14403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:37.151275 14403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 1 LEADER]: Becoming Leader. State: Replica: 29138508bd154071a1b3b5befcc2859c, State: Running, Role: LEADER
I20260807 05:42:37.151932 14403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER }
I20260807 05:42:37.152602 14400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:37.156205 14404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29138508bd154071a1b3b5befcc2859c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER } }
I20260807 05:42:37.156817 14404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:37.156502 14405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29138508bd154071a1b3b5befcc2859c. Latest consensus state: current_term: 1 leader_uuid: "29138508bd154071a1b3b5befcc2859c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29138508bd154071a1b3b5befcc2859c" member_type: VOTER } }
I20260807 05:42:37.157567 14405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:37.160305 14410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:37.168229 14410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:37.171275 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:37.179147 14410 catalog_manager.cc:1347] Generated new cluster ID: 152b4c13971d45f38a694a3246ba6f04
I20260807 05:42:37.179405 14410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:37.198447 14410 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:37.200299 14410 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:37.212307 14410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c: Generated new TSK 0
I20260807 05:42:37.213076 14410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:37.238462 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:37.244594 14421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.245378 14422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.247017 14424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.248024 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:37.248803 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:37.249089 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:37.249222 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081357249213 us; error 0 us; skew 500 ppm
I20260807 05:42:37.249733 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:37.252233 10684 webserver.cc:460] Webserver started at http://127.10.111.1:41007/ using document root <none> and password file <none>
I20260807 05:42:37.252665 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:37.252980 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:37.253262 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:37.254665 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "b1e9204d701c41958af6442a0e694415"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.259466 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:42:37.263054 14429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.264097 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:42:37.264407 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "b1e9204d701c41958af6442a0e694415"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.264708 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:37.287518 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:37.288576 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:37.293654 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:37.294013 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.294337 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:37.294543 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.352367 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:40955
I20260807 05:42:37.352502 14502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:40955 every 8 connection(s)
I20260807 05:42:37.357589 14503 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33985
I20260807 05:42:37.358110 14503 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:37.358918 14503 heartbeater.cc:511] Master 127.10.111.62:33985 requested a full tablet report, sending...
I20260807 05:42:37.361073 14362 ts_manager.cc:194] Registered new tserver with Master: b1e9204d701c41958af6442a0e694415 (127.10.111.1:40955)
I20260807 05:42:37.362113 10684 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006142727s
I20260807 05:42:37.363180 14362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38460
I20260807 05:42:37.399609 14468 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 05:42:37.400882 14468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:37.412778 14514 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: Bootstrap starting.
I20260807 05:42:37.417599 14514 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:37.422343 14514 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: No bootstrap required, opened a new log
I20260807 05:42:37.422806 14514 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 05:42:37.424899 14514 raft_consensus.cc:348] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.425388 14514 raft_consensus.cc:374] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:37.425664 14514 raft_consensus.cc:729] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e9204d701c41958af6442a0e694415, State: Initialized, Role: FOLLOWER
I20260807 05:42:37.426326 14514 consensus_queue.cc:260] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.426877 14514 raft_consensus.cc:388] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:37.427155 14514 raft_consensus.cc:482] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:37.427461 14514 raft_consensus.cc:3037] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:37.432360 14514 raft_consensus.cc:504] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.433056 14514 leader_election.cc:302] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e9204d701c41958af6442a0e694415; no voters:
I20260807 05:42:37.434350 14514 leader_election.cc:288] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:37.434716 14516 raft_consensus.cc:2781] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:37.436142 14516 raft_consensus.cc:686] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 1 LEADER]: Becoming Leader. State: Replica: b1e9204d701c41958af6442a0e694415, State: Running, Role: LEADER
I20260807 05:42:37.436477 14503 heartbeater.cc:503] Master 127.10.111.62:33985 was elected leader, sending a full tablet report...
I20260807 05:42:37.437065 14514 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 05:42:37.436926 14516 consensus_queue.cc:237] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.443049 14517 tablet_replica.cc:439] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1e9204d701c41958af6442a0e694415" committed_config { opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } } }
I20260807 05:42:37.443617 14517 tablet_replica.cc:442] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:37.443995 14518 tablet_replica.cc:439] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: TxnStatusTablet state changed. Reason: New leader b1e9204d701c41958af6442a0e694415. Latest consensus state: current_term: 1 leader_uuid: "b1e9204d701c41958af6442a0e694415" committed_config { opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } } }
I20260807 05:42:37.444530 14518 tablet_replica.cc:442] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:42:37.445384 14521 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
W20260807 05:42:37.445530 14362 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:37.445904 14521 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 05:42:37.449877 14362 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 05:42:37.473120 14468 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 05:42:37.474148 14468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:37.483700 14514 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: Bootstrap starting.
I20260807 05:42:37.487985 14514 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:37.492367 14514 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: No bootstrap required, opened a new log
I20260807 05:42:37.492827 14514 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.002s
I20260807 05:42:37.494724 14514 raft_consensus.cc:348] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.495134 14514 raft_consensus.cc:374] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:37.495404 14514 raft_consensus.cc:729] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e9204d701c41958af6442a0e694415, State: Initialized, Role: FOLLOWER
I20260807 05:42:37.496043 14514 consensus_queue.cc:260] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.496526 14514 raft_consensus.cc:388] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:37.496806 14514 raft_consensus.cc:482] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:37.497119 14514 raft_consensus.cc:3037] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:37.501703 14514 raft_consensus.cc:504] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
I20260807 05:42:37.502252 14514 leader_election.cc:302] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e9204d701c41958af6442a0e694415; no voters:
I20260807 05:42:37.502738 14514 leader_election.cc:288] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:37.502944 14517 raft_consensus.cc:2781] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:37.503455 14517 raft_consensus.cc:686] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 1 LEADER]: Becoming Leader. State: Replica: b1e9204d701c41958af6442a0e694415, State: Running, Role: LEADER
I20260807 05:42:37.504215 14514 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: Time spent starting tablet: real 0.011s user 0.007s sys 0.001s
I20260807 05:42:37.504143 14517 consensus_queue.cc:237] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "b1e9204d701c41958af6442a0e694415" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 40955 } }
W20260807 05:42:37.509636 14362 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260807 05:42:37.513865 14362 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260807 05:42:37.535261 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:37.557756 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:37.558689 10684 tablet_replica.cc:330] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415: stopping tablet replica
I20260807 05:42:37.559345 10684 raft_consensus.cc:2229] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:37.559860 10684 raft_consensus.cc:2258] T 22222222222222222222222222222222 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:37.562575 10684 tablet_replica.cc:330] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415: stopping tablet replica
I20260807 05:42:37.563158 10684 raft_consensus.cc:2229] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:37.563737 10684 raft_consensus.cc:2258] T 11111111111111111111111111111111 P b1e9204d701c41958af6442a0e694415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:37.586522 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:37.598718 10684 master.cc:537] Master@127.10.111.62:33985 shutting down...
I20260807 05:42:37.615566 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:37.616144 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:37.616623 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 29138508bd154071a1b3b5befcc2859c: stopping tablet replica
I20260807 05:42:37.626639 10684 master.cc:552] Master@127.10.111.62:33985 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (614 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3578 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260807 05:42:37.650475 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:40327
I20260807 05:42:37.651702 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:37.656960 14533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.658073 14534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.659794 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:37.659766 14536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.660948 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:37.661118 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:37.661301 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081357661288 us; error 0 us; skew 500 ppm
I20260807 05:42:37.661870 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:37.664273 10684 webserver.cc:460] Webserver started at http://127.10.111.62:36181/ using document root <none> and password file <none>
I20260807 05:42:37.664829 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:37.665031 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:37.665297 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:37.666495 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "683d95ec3a0e458f8fa872eb916cc0f0"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.671470 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:42:37.674947 14541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.675685 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:42:37.675977 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "683d95ec3a0e458f8fa872eb916cc0f0"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.676258 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:37.685460 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:37.686450 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:37.723030 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:40327
I20260807 05:42:37.723155 14598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:40327 every 8 connection(s)
I20260807 05:42:37.726800 14599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:37.737067 14599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0: Bootstrap starting.
I20260807 05:42:37.741168 14599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:37.745148 14599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0: No bootstrap required, opened a new log
I20260807 05:42:37.747051 14599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER }
I20260807 05:42:37.747507 14599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:37.747692 14599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683d95ec3a0e458f8fa872eb916cc0f0, State: Initialized, Role: FOLLOWER
I20260807 05:42:37.748312 14599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER }
I20260807 05:42:37.748804 14599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:37.749045 14599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:37.749310 14599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:37.753695 14599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER }
I20260807 05:42:37.754204 14599 leader_election.cc:302] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 683d95ec3a0e458f8fa872eb916cc0f0; no voters:
I20260807 05:42:37.755369 14599 leader_election.cc:288] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:37.755728 14602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:37.757177 14602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 1 LEADER]: Becoming Leader. State: Replica: 683d95ec3a0e458f8fa872eb916cc0f0, State: Running, Role: LEADER
I20260807 05:42:37.757840 14602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER }
I20260807 05:42:37.758384 14599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:37.760878 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 683d95ec3a0e458f8fa872eb916cc0f0. Latest consensus state: current_term: 1 leader_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER } }
I20260807 05:42:37.760831 14603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683d95ec3a0e458f8fa872eb916cc0f0" member_type: VOTER } }
I20260807 05:42:37.761632 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:37.762061 14603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:37.767611 14608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:37.772641 14608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:37.774016 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:37.781054 14608 catalog_manager.cc:1347] Generated new cluster ID: aeed99c187dc4d4abc28b88ee894c13c
I20260807 05:42:37.781352 14608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:37.803635 14608 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:37.805299 14608 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:37.815956 14608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0: Generated new TSK 0
I20260807 05:42:37.816646 14608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:37.840242 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:37.846403 14621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.847697 14622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.849882 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:37.850327 14624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.851322 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:37.851578 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:37.851740 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081357851728 us; error 0 us; skew 500 ppm
I20260807 05:42:37.852351 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:37.854828 10684 webserver.cc:460] Webserver started at http://127.10.111.1:42663/ using document root <none> and password file <none>
I20260807 05:42:37.855338 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:37.855540 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:37.855813 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:37.857050 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "3857fbdb9584435fa08d1ad0a70f5c14"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.861572 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 05:42:37.865033 14629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.865760 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:42:37.866061 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "3857fbdb9584435fa08d1ad0a70f5c14"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.866351 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:37.897994 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:37.899053 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:37.900504 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:37.902835 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:37.903052 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.903306 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:37.903481 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.943029 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:44559
I20260807 05:42:37.943148 14693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:44559 every 8 connection(s)
I20260807 05:42:37.947647 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:37.955169 14698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:37.955477 14699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.959720 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:37.959815 14701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:37.961083 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:37.961318 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:37.961519 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081357961501 us; error 0 us; skew 500 ppm
I20260807 05:42:37.962039 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:37.962961 14694 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40327
I20260807 05:42:37.963341 14694 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:37.964040 14694 heartbeater.cc:511] Master 127.10.111.62:40327 requested a full tablet report, sending...
I20260807 05:42:37.964996 10684 webserver.cc:460] Webserver started at http://127.10.111.2:40123/ using document root <none> and password file <none>
I20260807 05:42:37.965667 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:37.965901 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:37.966229 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:37.966212 14561 ts_manager.cc:194] Registered new tserver with Master: 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559)
I20260807 05:42:37.967886 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "e5d81341a6b64b7c84427cc4d41bee2c"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.969230 14561 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39338
I20260807 05:42:37.973351 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 05:42:37.977056 14706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:37.977965 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:42:37.978296 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "e5d81341a6b64b7c84427cc4d41bee2c"
format_stamp: "Formatted at 2026-08-07 05:42:37 on dist-test-slave-qg60"
I20260807 05:42:37.978606 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:38.016021 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:38.017213 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:38.018759 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:38.021118 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:38.021353 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:38.021615 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:38.021785 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:38.060210 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:42707
I20260807 05:42:38.060322 14771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:42707 every 8 connection(s)
I20260807 05:42:38.064913 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:38.071943 14775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:38.073196 14776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:38.076984 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:38.076902 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:38.078346 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:38.078593 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:38.078815 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081358078782 us; error 0 us; skew 500 ppm
I20260807 05:42:38.079470 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:38.081427 14772 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40327
I20260807 05:42:38.081857 14772 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:38.082394 10684 webserver.cc:460] Webserver started at http://127.10.111.3:32987/ using document root <none> and password file <none>
I20260807 05:42:38.082783 14772 heartbeater.cc:511] Master 127.10.111.62:40327 requested a full tablet report, sending...
I20260807 05:42:38.083055 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:38.083323 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:38.083636 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:38.084664 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-2-root/instance:
uuid: "dff2ced530344cf6885a34b37848769a"
format_stamp: "Formatted at 2026-08-07 05:42:38 on dist-test-slave-qg60"
I20260807 05:42:38.085218 14561 ts_manager.cc:194] Registered new tserver with Master: e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707)
I20260807 05:42:38.086824 14561 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39350
I20260807 05:42:38.089572 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:42:38.093051 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:38.093838 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:42:38.094162 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-2-root
uuid: "dff2ced530344cf6885a34b37848769a"
format_stamp: "Formatted at 2026-08-07 05:42:38 on dist-test-slave-qg60"
I20260807 05:42:38.094444 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:38.126266 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:38.127404 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:38.128891 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:38.131227 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:38.131460 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:38.131698 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:38.131867 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:38.171629 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.3:34449
I20260807 05:42:38.171727 14848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.3:34449 every 8 connection(s)
I20260807 05:42:38.187252 14849 heartbeater.cc:347] Connected to a master server at 127.10.111.62:40327
I20260807 05:42:38.187670 14849 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:38.188328 14849 heartbeater.cc:511] Master 127.10.111.62:40327 requested a full tablet report, sending...
I20260807 05:42:38.190240 14561 ts_manager.cc:194] Registered new tserver with Master: dff2ced530344cf6885a34b37848769a (127.10.111.3:34449)
I20260807 05:42:38.190980 10684 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015888659s
I20260807 05:42:38.192060 14561 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39356
I20260807 05:42:38.213369 10684 test_util.cc:268] Using random seed: 2028302561
I20260807 05:42:38.237629 14561 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39374:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:42:38.298645 14813 tablet_service.cc:1467] Processing CreateTablet for tablet 487021048116465e8308eda03d30cf5a (DEFAULT_TABLE table=test-workload [id=78b7eaded00a4336aa218778b54579e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:38.299955 14813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 487021048116465e8308eda03d30cf5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:38.302691 14736 tablet_service.cc:1467] Processing CreateTablet for tablet 487021048116465e8308eda03d30cf5a (DEFAULT_TABLE table=test-workload [id=78b7eaded00a4336aa218778b54579e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:38.303774 14736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 487021048116465e8308eda03d30cf5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:38.309617 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 487021048116465e8308eda03d30cf5a (DEFAULT_TABLE table=test-workload [id=78b7eaded00a4336aa218778b54579e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:38.311005 14657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 487021048116465e8308eda03d30cf5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:38.312316 14870 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: Bootstrap starting.
I20260807 05:42:38.318267 14870 tablet_bootstrap.cc:654] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:38.320150 14871 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Bootstrap starting.
I20260807 05:42:38.325426 14871 tablet_bootstrap.cc:654] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:38.325562 14870 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: No bootstrap required, opened a new log
I20260807 05:42:38.326088 14870 ts_tablet_manager.cc:1397] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260807 05:42:38.329396 14870 raft_consensus.cc:348] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.329947 14874 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Bootstrap starting.
I20260807 05:42:38.330233 14871 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: No bootstrap required, opened a new log
I20260807 05:42:38.330225 14870 raft_consensus.cc:374] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:38.330763 14870 raft_consensus.cc:729] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff2ced530344cf6885a34b37848769a, State: Initialized, Role: FOLLOWER
I20260807 05:42:38.330888 14871 ts_tablet_manager.cc:1397] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 05:42:38.331507 14870 consensus_queue.cc:260] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.333632 14871 raft_consensus.cc:348] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.334066 14849 heartbeater.cc:503] Master 127.10.111.62:40327 was elected leader, sending a full tablet report...
I20260807 05:42:38.334574 14870 ts_tablet_manager.cc:1428] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 05:42:38.335050 14871 raft_consensus.cc:374] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:38.335371 14871 raft_consensus.cc:729] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d81341a6b64b7c84427cc4d41bee2c, State: Initialized, Role: FOLLOWER
I20260807 05:42:38.336189 14871 consensus_queue.cc:260] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.336974 14874 tablet_bootstrap.cc:654] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:38.339612 14772 heartbeater.cc:503] Master 127.10.111.62:40327 was elected leader, sending a full tablet report...
I20260807 05:42:38.341010 14871 ts_tablet_manager.cc:1428] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 05:42:38.344219 14874 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: No bootstrap required, opened a new log
I20260807 05:42:38.344695 14874 ts_tablet_manager.cc:1397] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260807 05:42:38.347373 14874 raft_consensus.cc:348] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.348217 14874 raft_consensus.cc:374] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:38.348541 14874 raft_consensus.cc:729] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3857fbdb9584435fa08d1ad0a70f5c14, State: Initialized, Role: FOLLOWER
I20260807 05:42:38.349499 14874 consensus_queue.cc:260] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.352221 14694 heartbeater.cc:503] Master 127.10.111.62:40327 was elected leader, sending a full tablet report...
I20260807 05:42:38.352479 14874 ts_tablet_manager.cc:1428] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:42:38.356460 14876 raft_consensus.cc:482] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:42:38.356961 14876 raft_consensus.cc:504] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.359191 14876 leader_election.cc:288] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559), dff2ced530344cf6885a34b37848769a (127.10.111.3:34449)
I20260807 05:42:38.374821 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" is_pre_election: true
I20260807 05:42:38.375748 14669 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5d81341a6b64b7c84427cc4d41bee2c in term 0.
I20260807 05:42:38.377213 14708 leader_election.cc:302] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3857fbdb9584435fa08d1ad0a70f5c14, e5d81341a6b64b7c84427cc4d41bee2c; no voters:
I20260807 05:42:38.378104 14876 raft_consensus.cc:2781] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:42:38.378494 14876 raft_consensus.cc:482] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:42:38.378445 14824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff2ced530344cf6885a34b37848769a" is_pre_election: true
I20260807 05:42:38.379098 14876 raft_consensus.cc:3037] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:38.379554 14824 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5d81341a6b64b7c84427cc4d41bee2c in term 0.
I20260807 05:42:38.384419 14876 raft_consensus.cc:504] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.386302 14876 leader_election.cc:288] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [CANDIDATE]: Term 1 election: Requested vote from peers 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559), dff2ced530344cf6885a34b37848769a (127.10.111.3:34449)
I20260807 05:42:38.387364 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3857fbdb9584435fa08d1ad0a70f5c14"
I20260807 05:42:38.387393 14824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff2ced530344cf6885a34b37848769a"
I20260807 05:42:38.387977 14669 raft_consensus.cc:3037] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:38.388113 14824 raft_consensus.cc:3037] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:38.393688 14669 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5d81341a6b64b7c84427cc4d41bee2c in term 1.
I20260807 05:42:38.393947 14824 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5d81341a6b64b7c84427cc4d41bee2c in term 1.
I20260807 05:42:38.394693 14708 leader_election.cc:302] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3857fbdb9584435fa08d1ad0a70f5c14, e5d81341a6b64b7c84427cc4d41bee2c; no voters:
I20260807 05:42:38.395578 14876 raft_consensus.cc:2781] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:38.396629 14876 raft_consensus.cc:686] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 1 LEADER]: Becoming Leader. State: Replica: e5d81341a6b64b7c84427cc4d41bee2c, State: Running, Role: LEADER
I20260807 05:42:38.397359 14876 consensus_queue.cc:237] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:38.404835 14561 catalog_manager.cc:5495] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2). New cstate: current_term: 1 leader_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } } }
I20260807 05:42:38.690045 14657 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:38.691412 14736 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:38.692170 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:38.779282 14876 consensus_queue.cc:1035] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 05:42:38.837420 14876 consensus_queue.cc:1035] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 0 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 487021048116465e8308eda03d30cf5a of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559): RUNNING
dff2ced530344cf6885a34b37848769a (127.10.111.3:34449): RUNNING
e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): RUNNING [LEADER]
All reported replicas are:
A = 3857fbdb9584435fa08d1ad0a70f5c14
B = dff2ced530344cf6885a34b37848769a
C = e5d81341a6b64b7c84427cc4d41bee2c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:38.933254 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:39.301941 14657 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:39.304422 14736 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:39.307181 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 0 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 05:42:39.500084 10684 log_verifier.cc:126] Checking tablet 487021048116465e8308eda03d30cf5a
I20260807 05:42:39.563350 10684 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 487021048116465e8308eda03d30cf5a
I20260807 05:42:39.563915 10684 tablet_replica.cc:330] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: stopping tablet replica
I20260807 05:42:39.564399 10684 raft_consensus.cc:2229] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:39.565256 10684 raft_consensus.cc:2258] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:39.781595 14657 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:39.783684 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:39.785383 14736 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 0 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 487021048116465e8308eda03d30cf5a of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559): RUNNING
dff2ced530344cf6885a34b37848769a (127.10.111.3:34449): RUNNING
e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = 3857fbdb9584435fa08d1ad0a70f5c14
B = dff2ced530344cf6885a34b37848769a
C = e5d81341a6b64b7c84427cc4d41bee2c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:39.983207 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:40.351837 14657 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:40.356489 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:40.356691 14736 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 0 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 487021048116465e8308eda03d30cf5a of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559): RUNNING
dff2ced530344cf6885a34b37848769a (127.10.111.3:34449): RUNNING
e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = 3857fbdb9584435fa08d1ad0a70f5c14
B = dff2ced530344cf6885a34b37848769a
C = e5d81341a6b64b7c84427cc4d41bee2c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:40.627682 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:40.801419 15000 raft_consensus.cc:482] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5d81341a6b64b7c84427cc4d41bee2c)
I20260807 05:42:40.802099 15000 raft_consensus.cc:504] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:40.804387 15000 leader_election.cc:288] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559), e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707)
I20260807 05:42:40.822291 14746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "dff2ced530344cf6885a34b37848769a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" is_pre_election: true
I20260807 05:42:40.823060 14668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "dff2ced530344cf6885a34b37848769a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" is_pre_election: true
W20260807 05:42:40.823997 14784 leader_election.cc:341] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): Illegal state: cannot vote while shut down
I20260807 05:42:40.824869 14785 leader_election.cc:302] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dff2ced530344cf6885a34b37848769a; no voters: 3857fbdb9584435fa08d1ad0a70f5c14, e5d81341a6b64b7c84427cc4d41bee2c
I20260807 05:42:40.825563 15000 raft_consensus.cc:2726] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:42:40.842517 15005 raft_consensus.cc:482] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5d81341a6b64b7c84427cc4d41bee2c)
I20260807 05:42:40.842965 15005 raft_consensus.cc:504] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:40.845099 15005 leader_election.cc:288] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dff2ced530344cf6885a34b37848769a (127.10.111.3:34449), e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707)
I20260807 05:42:40.860643 14824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dff2ced530344cf6885a34b37848769a" is_pre_election: true
I20260807 05:42:40.861510 14824 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3857fbdb9584435fa08d1ad0a70f5c14 in term 1.
I20260807 05:42:40.863042 14746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" is_pre_election: true
I20260807 05:42:40.863117 14630 leader_election.cc:302] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3857fbdb9584435fa08d1ad0a70f5c14, dff2ced530344cf6885a34b37848769a; no voters:
W20260807 05:42:40.864496 14630 leader_election.cc:341] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): Illegal state: cannot vote while shut down
I20260807 05:42:40.864576 15005 raft_consensus.cc:2781] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:42:40.865129 15005 raft_consensus.cc:482] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5d81341a6b64b7c84427cc4d41bee2c)
I20260807 05:42:40.865489 15005 raft_consensus.cc:3037] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:42:40.875043 15005 raft_consensus.cc:504] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:40.876648 15005 leader_election.cc:288] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 election: Requested vote from peers dff2ced530344cf6885a34b37848769a (127.10.111.3:34449), e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707)
I20260807 05:42:40.877586 14824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dff2ced530344cf6885a34b37848769a"
I20260807 05:42:40.877825 14746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "487021048116465e8308eda03d30cf5a" candidate_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5d81341a6b64b7c84427cc4d41bee2c"
I20260807 05:42:40.878257 14824 raft_consensus.cc:3037] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 1 FOLLOWER]: Advancing to term 2
W20260807 05:42:40.878988 14630 leader_election.cc:341] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): Illegal state: cannot vote while shut down
I20260807 05:42:40.883777 14824 raft_consensus.cc:2454] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3857fbdb9584435fa08d1ad0a70f5c14 in term 2.
I20260807 05:42:40.884752 14630 leader_election.cc:302] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3857fbdb9584435fa08d1ad0a70f5c14, dff2ced530344cf6885a34b37848769a; no voters: e5d81341a6b64b7c84427cc4d41bee2c
I20260807 05:42:40.885555 15005 raft_consensus.cc:2781] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:42:40.887187 15005 raft_consensus.cc:686] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 LEADER]: Becoming Leader. State: Replica: 3857fbdb9584435fa08d1ad0a70f5c14, State: Running, Role: LEADER
I20260807 05:42:40.888108 15005 consensus_queue.cc:237] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:40.896965 14559 catalog_manager.cc:5495] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 reported cstate change: term changed from 1 to 2, leader changed from e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) to 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1). New cstate: current_term: 2 leader_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } } }
I20260807 05:42:41.100912 14657 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:41.101148 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:41.107148 14736 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 1 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 487021048116465e8308eda03d30cf5a of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559): RUNNING [LEADER]
dff2ced530344cf6885a34b37848769a (127.10.111.3:34449): RUNNING
e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = 3857fbdb9584435fa08d1ad0a70f5c14
B = dff2ced530344cf6885a34b37848769a
C = e5d81341a6b64b7c84427cc4d41bee2c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | -1 | Yes
B | A B C | 2 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:41.303658 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:41.425959 14824 raft_consensus.cc:1261] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Refusing update from remote peer 3857fbdb9584435fa08d1ad0a70f5c14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 05:42:41.427383 15041 consensus_queue.cc:1035] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 05:42:41.440110 14630 consensus_peers.cc:487] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 -> Peer e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): Couldn't send request to peer e5d81341a6b64b7c84427cc4d41bee2c. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:42:41.440600 15044 raft_consensus.cc:998] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Attempting to remove follower e5d81341a6b64b7c84427cc4d41bee2c from the Raft config. Reason: The tablet replica hosted on peer e5d81341a6b64b7c84427cc4d41bee2c has failed
I20260807 05:42:41.443010 15044 consensus_queue.cc:237] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } }
I20260807 05:42:41.447041 14824 raft_consensus.cc:1261] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Refusing update from remote peer 3857fbdb9584435fa08d1ad0a70f5c14: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 05:42:41.448251 15041 consensus_queue.cc:1035] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 05:42:41.453600 15044 raft_consensus.cc:2932] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } }
I20260807 05:42:41.455089 14824 raft_consensus.cc:2932] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } }
I20260807 05:42:41.464103 14559 catalog_manager.cc:5495] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 reported cstate change: config changed from index -1 to 3, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) evicted. New cstate: current_term: 2 leader_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } }
I20260807 05:42:41.476121 14736 tablet_service.cc:1514] Processing DeleteTablet for tablet 487021048116465e8308eda03d30cf5a with delete_type TABLET_DATA_TOMBSTONED (TS e5d81341a6b64b7c84427cc4d41bee2c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:58444
I20260807 05:42:41.478688 15050 ts_tablet_manager.cc:1905] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 05:42:41.479961 14668 consensus_queue.cc:237] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:41.485493 14824 raft_consensus.cc:1261] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Refusing update from remote peer 3857fbdb9584435fa08d1ad0a70f5c14: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260807 05:42:41.485625 14630 consensus_peers.cc:487] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 -> Peer e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): Couldn't send request to peer e5d81341a6b64b7c84427cc4d41bee2c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:42:41.486773 15044 consensus_queue.cc:1035] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:42:41.492383 15041 raft_consensus.cc:2932] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } } }
I20260807 05:42:41.493620 15050 ts_tablet_manager.cc:1918] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260807 05:42:41.494088 15050 log.cc:1196] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786081299864053-10684-0/minicluster-data/ts-1-root/wals/487021048116465e8308eda03d30cf5a
I20260807 05:42:41.494179 14824 raft_consensus.cc:2932] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } } }
I20260807 05:42:41.496053 14542 catalog_manager.cc:4841] TS e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): tablet 487021048116465e8308eda03d30cf5a (table test-workload [id=78b7eaded00a4336aa218778b54579e6]) successfully deleted
I20260807 05:42:41.502265 14544 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet 487021048116465e8308eda03d30cf5a with cas_config_opid_index 3: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260807 05:42:41.505599 14559 catalog_manager.cc:5495] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 reported cstate change: config changed from index 3 to 4, VOTER e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2) added. New cstate: current_term: 2 leader_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } } }
I20260807 05:42:41.857554 14657 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:41.862498 14736 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:41.863590 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:41.992599 15082 ts_tablet_manager.cc:927] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Initiating tablet copy from peer 3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559)
I20260807 05:42:42.001664 15082 tablet_copy_client.cc:316] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet copy: Beginning tablet copy session from remote peer at address 127.10.111.1:44559
I20260807 05:42:42.003787 14679 tablet_copy_service.cc:140] P 3857fbdb9584435fa08d1ad0a70f5c14: Received BeginTabletCopySession request for tablet 487021048116465e8308eda03d30cf5a from peer e5d81341a6b64b7c84427cc4d41bee2c ({username='slave'} at 127.0.0.1:52078)
I20260807 05:42:42.004356 14679 tablet_copy_service.cc:161] P 3857fbdb9584435fa08d1ad0a70f5c14: Beginning new tablet copy session on tablet 487021048116465e8308eda03d30cf5a from peer e5d81341a6b64b7c84427cc4d41bee2c at {username='slave'} at 127.0.0.1:52078: session id = e5d81341a6b64b7c84427cc4d41bee2c-487021048116465e8308eda03d30cf5a
I20260807 05:42:42.013079 14679 tablet_copy_source_session.cc:205] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:42:42.016088 15082 ts_tablet_manager.cc:1905] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260807 05:42:42.031656 15082 ts_tablet_manager.cc:1918] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260807 05:42:42.032414 15082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 487021048116465e8308eda03d30cf5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:42.039856 15082 tablet_copy_client.cc:791] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet copy: Starting download of 0 data blocks...
I20260807 05:42:42.040423 15082 tablet_copy_client.cc:655] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet copy: Starting download of 1 WAL segments...
I20260807 05:42:42.044399 15082 tablet_copy_client.cc:531] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:42:42.052295 15082 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Bootstrap starting.
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 1 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 487021048116465e8308eda03d30cf5a of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
3857fbdb9584435fa08d1ad0a70f5c14 (127.10.111.1:44559): RUNNING [LEADER]
dff2ced530344cf6885a34b37848769a (127.10.111.3:34449): RUNNING
e5d81341a6b64b7c84427cc4d41bee2c (127.10.111.2:42707): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = 3857fbdb9584435fa08d1ad0a70f5c14
B = dff2ced530344cf6885a34b37848769a
C = e5d81341a6b64b7c84427cc4d41bee2c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 4 | Yes
B | A* B C | 2 | 4 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:42.066215 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:42.076845 15082 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:42.077900 15082 tablet_bootstrap.cc:492] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Bootstrap complete.
I20260807 05:42:42.078518 15082 ts_tablet_manager.cc:1397] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.010s
I20260807 05:42:42.080497 15082 raft_consensus.cc:348] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:42.081164 15082 raft_consensus.cc:729] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d81341a6b64b7c84427cc4d41bee2c, State: Initialized, Role: FOLLOWER
I20260807 05:42:42.081689 15082 consensus_queue.cc:260] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3857fbdb9584435fa08d1ad0a70f5c14" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 44559 } } peers { permanent_uuid: "dff2ced530344cf6885a34b37848769a" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34449 } } peers { permanent_uuid: "e5d81341a6b64b7c84427cc4d41bee2c" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 42707 } }
I20260807 05:42:42.083827 15082 ts_tablet_manager.cc:1428] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 05:42:42.085584 14679 tablet_copy_service.cc:342] P 3857fbdb9584435fa08d1ad0a70f5c14: Request end of tablet copy session e5d81341a6b64b7c84427cc4d41bee2c-487021048116465e8308eda03d30cf5a received from {username='slave'} at 127.0.0.1:52078
I20260807 05:42:42.085973 14679 tablet_copy_service.cc:433] P 3857fbdb9584435fa08d1ad0a70f5c14: ending tablet copy session e5d81341a6b64b7c84427cc4d41bee2c-487021048116465e8308eda03d30cf5a on tablet 487021048116465e8308eda03d30cf5a with peer e5d81341a6b64b7c84427cc4d41bee2c
I20260807 05:42:42.483421 14746 raft_consensus.cc:1203] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.3->[2.4-2.4] Dedup: 2.4->[]
I20260807 05:42:42.801007 14657 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:42.807126 14736 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:42.809995 14813 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
683d95ec3a0e458f8fa872eb916cc0f0 | 127.10.111.62:40327 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3857fbdb9584435fa08d1ad0a70f5c14 | 127.10.111.1:44559 | HEALTHY | <none> | 1 | 0
dff2ced530344cf6885a34b37848769a | 127.10.111.3:34449 | HEALTHY | <none> | 0 | 0
e5d81341a6b64b7c84427cc4d41bee2c | 127.10.111.2:42707 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 05:42:42.983763 10684 log_verifier.cc:126] Checking tablet 487021048116465e8308eda03d30cf5a
I20260807 05:42:43.043620 10684 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 487021048116465e8308eda03d30cf5a
I20260807 05:42:43.047350 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:43.079694 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:43.080510 10684 tablet_replica.cc:330] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14: stopping tablet replica
I20260807 05:42:43.081125 10684 raft_consensus.cc:2229] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 LEADER]: Raft consensus shutting down.
I20260807 05:42:43.081925 10684 raft_consensus.cc:2258] T 487021048116465e8308eda03d30cf5a P 3857fbdb9584435fa08d1ad0a70f5c14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:43.102700 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:43.114156 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:42:43.136162 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:43.136953 10684 tablet_replica.cc:330] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c: stopping tablet replica
I20260807 05:42:43.137562 10684 raft_consensus.cc:2229] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:43.137967 10684 raft_consensus.cc:2258] T 487021048116465e8308eda03d30cf5a P e5d81341a6b64b7c84427cc4d41bee2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:43.158090 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:42:43.169997 10684 tablet_server.cc:178] TabletServer@127.10.111.3:0 shutting down...
I20260807 05:42:43.193375 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:43.194103 10684 tablet_replica.cc:330] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a: stopping tablet replica
I20260807 05:42:43.194659 10684 raft_consensus.cc:2229] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:43.195209 10684 raft_consensus.cc:2258] T 487021048116465e8308eda03d30cf5a P dff2ced530344cf6885a34b37848769a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:43.215873 10684 tablet_server.cc:194] TabletServer@127.10.111.3:0 shutdown complete.
I20260807 05:42:43.226325 10684 master.cc:537] Master@127.10.111.62:40327 shutting down...
I20260807 05:42:43.245813 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:43.246305 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:43.246610 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 683d95ec3a0e458f8fa872eb916cc0f0: stopping tablet replica
I20260807 05:42:43.264336 10684 master.cc:552] Master@127.10.111.62:40327 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (5646 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260807 05:42:43.297066 10684 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.111.62:33399
I20260807 05:42:43.298106 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:43.303325 15120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.304275 15121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.306705 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:43.306720 15123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.307775 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:43.308000 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:43.308164 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081363308139 us; error 0 us; skew 500 ppm
I20260807 05:42:43.308678 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:43.311094 10684 webserver.cc:460] Webserver started at http://127.10.111.62:41939/ using document root <none> and password file <none>
I20260807 05:42:43.311543 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:43.311717 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:43.311985 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:43.313133 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/master-0-root/instance:
uuid: "3518d7c2b06e475da793e58c284015ac"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.317492 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:42:43.320827 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.321543 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:42:43.321818 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/master-0-root
uuid: "3518d7c2b06e475da793e58c284015ac"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.322086 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:43.345908 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:43.347195 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:43.386830 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.62:33399
I20260807 05:42:43.386940 15186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.62:33399 every 8 connection(s)
I20260807 05:42:43.390614 15187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:43.401011 15187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac: Bootstrap starting.
I20260807 05:42:43.405350 15187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:43.409349 15187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac: No bootstrap required, opened a new log
I20260807 05:42:43.411244 15187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER }
I20260807 05:42:43.411607 15187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:43.411785 15187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3518d7c2b06e475da793e58c284015ac, State: Initialized, Role: FOLLOWER
I20260807 05:42:43.412246 15187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER }
I20260807 05:42:43.412657 15187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:42:43.412904 15187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:42:43.413132 15187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:43.417737 15187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER }
I20260807 05:42:43.418231 15187 leader_election.cc:302] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3518d7c2b06e475da793e58c284015ac; no voters:
I20260807 05:42:43.419445 15187 leader_election.cc:288] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:42:43.419911 15190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:43.421353 15190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 1 LEADER]: Becoming Leader. State: Replica: 3518d7c2b06e475da793e58c284015ac, State: Running, Role: LEADER
I20260807 05:42:43.422044 15190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER }
I20260807 05:42:43.422488 15187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:42:43.425084 15192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3518d7c2b06e475da793e58c284015ac. Latest consensus state: current_term: 1 leader_uuid: "3518d7c2b06e475da793e58c284015ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER } }
I20260807 05:42:43.425199 15191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3518d7c2b06e475da793e58c284015ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3518d7c2b06e475da793e58c284015ac" member_type: VOTER } }
I20260807 05:42:43.425855 15191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:43.425868 15192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [sys.catalog]: This master's current role is: LEADER
I20260807 05:42:43.431645 15195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:42:43.436585 15195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:42:43.439141 10684 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 05:42:43.445076 15195 catalog_manager.cc:1347] Generated new cluster ID: bea15cd43e974112ac2709e391f8697d
I20260807 05:42:43.445323 15195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:42:43.479630 15195 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:42:43.481396 15195 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:42:43.502130 15195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac: Generated new TSK 0
I20260807 05:42:43.502831 15195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:42:43.570817 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:43.577553 15208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.579437 15209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.580534 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.581351 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:43.582351 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:43.582573 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:43.582736 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081363582721 us; error 0 us; skew 500 ppm
I20260807 05:42:43.583267 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:43.585673 10684 webserver.cc:460] Webserver started at http://127.10.111.1:43367/ using document root <none> and password file <none>
I20260807 05:42:43.586151 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:43.586351 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:43.586615 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:43.587721 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-0-root/instance:
uuid: "ac8cb44d463a457f98f4ca3611f4f094"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.592141 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 05:42:43.595603 15216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.596345 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:42:43.596642 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-0-root
uuid: "ac8cb44d463a457f98f4ca3611f4f094"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.596966 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:43.609027 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:43.610160 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:43.611605 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:43.614042 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:43.614295 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.614523 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:43.614681 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.657043 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.1:33125
I20260807 05:42:43.657205 15283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.1:33125 every 8 connection(s)
I20260807 05:42:43.661689 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:43.669520 15288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.672618 15289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.673327 10684 server_base.cc:1034] running on GCE node
W20260807 05:42:43.674180 15291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.675384 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:43.675642 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:43.675840 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081363675822 us; error 0 us; skew 500 ppm
I20260807 05:42:43.676470 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:43.677686 15284 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33399
I20260807 05:42:43.678107 15284 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:43.678857 15284 heartbeater.cc:511] Master 127.10.111.62:33399 requested a full tablet report, sending...
I20260807 05:42:43.679456 10684 webserver.cc:460] Webserver started at http://127.10.111.2:43005/ using document root <none> and password file <none>
I20260807 05:42:43.679978 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:43.680217 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:43.680531 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:43.681262 15146 ts_manager.cc:194] Registered new tserver with Master: ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1:33125)
I20260807 05:42:43.682045 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-1-root/instance:
uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.684090 15146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47774
I20260807 05:42:43.688872 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 05:42:43.692269 15296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.693265 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:42:43.693555 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-1-root
uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.693836 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:43.708534 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:43.709771 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:43.711181 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:43.713444 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:43.713653 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.713888 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:43.714046 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.754004 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.2:46309
I20260807 05:42:43.754114 15362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.2:46309 every 8 connection(s)
I20260807 05:42:43.758873 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:43.766486 15367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.768940 15368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.771116 15370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.771322 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:43.772373 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:43.772629 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:43.772845 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081363772830 us; error 0 us; skew 500 ppm
I20260807 05:42:43.773500 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:43.774266 15363 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33399
I20260807 05:42:43.774643 15363 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:43.775321 15363 heartbeater.cc:511] Master 127.10.111.62:33399 requested a full tablet report, sending...
I20260807 05:42:43.776405 10684 webserver.cc:460] Webserver started at http://127.10.111.3:45569/ using document root <none> and password file <none>
I20260807 05:42:43.777046 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:43.777287 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:43.777598 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:43.777523 15146 ts_manager.cc:194] Registered new tserver with Master: 0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309)
I20260807 05:42:43.779181 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root/instance:
uuid: "27c89fae85e14964b2ecba095ce730d2"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.779904 15146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47786
I20260807 05:42:43.783943 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 05:42:43.787446 15375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.788211 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:42:43.788501 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root
uuid: "27c89fae85e14964b2ecba095ce730d2"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.788828 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:43.801889 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:43.802984 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:43.804375 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:43.806653 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:43.806860 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.807101 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:43.807263 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.849380 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.3:34103
I20260807 05:42:43.849503 15440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.3:34103 every 8 connection(s)
I20260807 05:42:43.854211 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:42:43.861630 15444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.864264 15445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:42:43.865849 15447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:42:43.866307 10684 server_base.cc:1034] running on GCE node
I20260807 05:42:43.867576 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 05:42:43.867870 10684 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 05:42:43.868098 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786081363868079 us; error 0 us; skew 500 ppm
I20260807 05:42:43.868709 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 05:42:43.869297 15441 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33399
I20260807 05:42:43.869804 15441 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:43.870641 15441 heartbeater.cc:511] Master 127.10.111.62:33399 requested a full tablet report, sending...
I20260807 05:42:43.871814 10684 webserver.cc:460] Webserver started at http://127.10.111.4:44159/ using document root <none> and password file <none>
I20260807 05:42:43.872321 10684 fs_manager.cc:359] Metadata directory not provided
I20260807 05:42:43.872526 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:42:43.872844 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:42:43.872810 15146 ts_manager.cc:194] Registered new tserver with Master: 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103)
I20260807 05:42:43.874123 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-3-root/instance:
uuid: "549528e61e034244812eae633e7e3971"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.874856 15146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47802
I20260807 05:42:43.878772 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 05:42:43.882330 15453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.883159 10684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:42:43.883446 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-3-root
uuid: "549528e61e034244812eae633e7e3971"
format_stamp: "Formatted at 2026-08-07 05:42:43 on dist-test-slave-qg60"
I20260807 05:42:43.883773 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:42:43.895733 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:42:43.896906 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:42:43.898450 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:42:43.901119 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:42:43.901336 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.901563 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:42:43.901728 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:42:43.945521 10684 rpc_server.cc:307] RPC server started. Bound to: 127.10.111.4:38321
I20260807 05:42:43.945665 15517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.111.4:38321 every 8 connection(s)
I20260807 05:42:43.961899 15518 heartbeater.cc:347] Connected to a master server at 127.10.111.62:33399
I20260807 05:42:43.962311 15518 heartbeater.cc:464] Registering TS with master...
I20260807 05:42:43.963114 15518 heartbeater.cc:511] Master 127.10.111.62:33399 requested a full tablet report, sending...
I20260807 05:42:44.055747 15146 ts_manager.cc:194] Registered new tserver with Master: 549528e61e034244812eae633e7e3971 (127.10.111.4:38321)
I20260807 05:42:44.058663 15146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47818
I20260807 05:42:44.069351 10684 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.120670823s
I20260807 05:42:44.204025 10684 test_util.cc:268] Using random seed: 2034293217
I20260807 05:42:44.229079 15146 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47834:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:42:44.294746 15246 tablet_service.cc:1467] Processing CreateTablet for tablet 44e7ee23ffe94dc2b64d9bc396690267 (DEFAULT_TABLE table=test-workload [id=2f62d9b4861a41b89da3fff0fa3b0e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:44.296106 15246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e7ee23ffe94dc2b64d9bc396690267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:44.297662 15326 tablet_service.cc:1467] Processing CreateTablet for tablet 44e7ee23ffe94dc2b64d9bc396690267 (DEFAULT_TABLE table=test-workload [id=2f62d9b4861a41b89da3fff0fa3b0e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:44.298712 15326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e7ee23ffe94dc2b64d9bc396690267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:44.303013 15404 tablet_service.cc:1467] Processing CreateTablet for tablet 44e7ee23ffe94dc2b64d9bc396690267 (DEFAULT_TABLE table=test-workload [id=2f62d9b4861a41b89da3fff0fa3b0e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:42:44.304333 15404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e7ee23ffe94dc2b64d9bc396690267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:44.313956 15542 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: Bootstrap starting.
I20260807 05:42:44.315548 15543 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: Bootstrap starting.
I20260807 05:42:44.320645 15542 tablet_bootstrap.cc:654] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:44.321964 15543 tablet_bootstrap.cc:654] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:44.322773 15544 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Bootstrap starting.
I20260807 05:42:44.329137 15542 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: No bootstrap required, opened a new log
I20260807 05:42:44.329607 15542 ts_tablet_manager.cc:1397] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260807 05:42:44.330834 15544 tablet_bootstrap.cc:654] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Neither blocks nor log segments found. Creating new log.
I20260807 05:42:44.331081 15543 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: No bootstrap required, opened a new log
I20260807 05:42:44.331604 15543 ts_tablet_manager.cc:1397] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260807 05:42:44.331818 15542 raft_consensus.cc:348] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.332617 15542 raft_consensus.cc:374] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:44.332939 15542 raft_consensus.cc:729] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8cb44d463a457f98f4ca3611f4f094, State: Initialized, Role: FOLLOWER
I20260807 05:42:44.333792 15542 consensus_queue.cc:260] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.334928 15543 raft_consensus.cc:348] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.335842 15543 raft_consensus.cc:374] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:44.336213 15543 raft_consensus.cc:729] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9b2b8c8da54d1c9f71f932bd56ee30, State: Initialized, Role: FOLLOWER
I20260807 05:42:44.337073 15542 ts_tablet_manager.cc:1428] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260807 05:42:44.337589 15544 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: No bootstrap required, opened a new log
I20260807 05:42:44.337013 15543 consensus_queue.cc:260] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.338035 15544 ts_tablet_manager.cc:1397] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260807 05:42:44.338112 15284 heartbeater.cc:503] Master 127.10.111.62:33399 was elected leader, sending a full tablet report...
I20260807 05:42:44.339841 15363 heartbeater.cc:503] Master 127.10.111.62:33399 was elected leader, sending a full tablet report...
I20260807 05:42:44.340375 15543 ts_tablet_manager.cc:1428] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 05:42:44.341321 15544 raft_consensus.cc:348] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.342041 15544 raft_consensus.cc:374] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:42:44.342339 15544 raft_consensus.cc:729] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c89fae85e14964b2ecba095ce730d2, State: Initialized, Role: FOLLOWER
I20260807 05:42:44.343048 15544 consensus_queue.cc:260] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.345839 15441 heartbeater.cc:503] Master 127.10.111.62:33399 was elected leader, sending a full tablet report...
I20260807 05:42:44.346174 15544 ts_tablet_manager.cc:1428] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 05:42:44.492296 15548 raft_consensus.cc:482] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:42:44.492877 15548 raft_consensus.cc:504] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.495025 15548 leader_election.cc:288] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309), 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103)
I20260807 05:42:44.512602 15338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e7ee23ffe94dc2b64d9bc396690267" candidate_uuid: "ac8cb44d463a457f98f4ca3611f4f094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" is_pre_election: true
I20260807 05:42:44.512689 15415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e7ee23ffe94dc2b64d9bc396690267" candidate_uuid: "ac8cb44d463a457f98f4ca3611f4f094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c89fae85e14964b2ecba095ce730d2" is_pre_election: true
I20260807 05:42:44.513382 15338 raft_consensus.cc:2454] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac8cb44d463a457f98f4ca3611f4f094 in term 0.
I20260807 05:42:44.513422 15415 raft_consensus.cc:2454] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac8cb44d463a457f98f4ca3611f4f094 in term 0.
I20260807 05:42:44.514492 15218 leader_election.cc:302] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9b2b8c8da54d1c9f71f932bd56ee30, ac8cb44d463a457f98f4ca3611f4f094; no voters:
I20260807 05:42:44.515216 15548 raft_consensus.cc:2781] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:42:44.515573 15548 raft_consensus.cc:482] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:42:44.515864 15548 raft_consensus.cc:3037] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:44.520912 15548 raft_consensus.cc:504] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.522404 15548 leader_election.cc:288] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [CANDIDATE]: Term 1 election: Requested vote from peers 0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309), 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103)
I20260807 05:42:44.523365 15415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e7ee23ffe94dc2b64d9bc396690267" candidate_uuid: "ac8cb44d463a457f98f4ca3611f4f094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c89fae85e14964b2ecba095ce730d2"
I20260807 05:42:44.523388 15338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e7ee23ffe94dc2b64d9bc396690267" candidate_uuid: "ac8cb44d463a457f98f4ca3611f4f094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30"
I20260807 05:42:44.524151 15415 raft_consensus.cc:3037] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:44.524233 15338 raft_consensus.cc:3037] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:42:44.529594 15338 raft_consensus.cc:2454] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8cb44d463a457f98f4ca3611f4f094 in term 1.
I20260807 05:42:44.529608 15415 raft_consensus.cc:2454] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8cb44d463a457f98f4ca3611f4f094 in term 1.
I20260807 05:42:44.530752 15218 leader_election.cc:302] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9b2b8c8da54d1c9f71f932bd56ee30, ac8cb44d463a457f98f4ca3611f4f094; no voters:
I20260807 05:42:44.531456 15548 raft_consensus.cc:2781] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:42:44.532439 15548 raft_consensus.cc:686] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 LEADER]: Becoming Leader. State: Replica: ac8cb44d463a457f98f4ca3611f4f094, State: Running, Role: LEADER
I20260807 05:42:44.533407 15548 consensus_queue.cc:237] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:44.541224 15146 catalog_manager.cc:5495] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1). New cstate: current_term: 1 leader_uuid: "ac8cb44d463a457f98f4ca3611f4f094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } health_report { overall_health: HEALTHY } } }
I20260807 05:42:44.829141 15326 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:44.842046 15404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:44.846944 15481 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:44.854734 15246 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:45.001330 15548 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:42:45.042138 15553 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27c89fae85e14964b2ecba095ce730d2" member_type: VOTER last_known_addr { host: "127.10.111.3" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:42:45.063352 15518 heartbeater.cc:503] Master 127.10.111.62:33399 was elected leader, sending a full tablet report...
W20260807 05:42:45.088541 15280 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8a20 after lost signal to thread 10687
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
3518d7c2b06e475da793e58c284015ac | 127.10.111.62:33399 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0c9b2b8c8da54d1c9f71f932bd56ee30 | 127.10.111.2:46309 | HEALTHY | <none> | 0 | 0
27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | HEALTHY | <none> | 0 | 0
549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | HEALTHY | <none> | 0 | 0
ac8cb44d463a457f98f4ca3611f4f094 | 127.10.111.1:33125 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 44e7ee23ffe94dc2b64d9bc396690267 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309): RUNNING
27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103): RUNNING
ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1:33125): RUNNING [LEADER]
All reported replicas are:
A = 0c9b2b8c8da54d1c9f71f932bd56ee30
B = 27c89fae85e14964b2ecba095ce730d2
C = ac8cb44d463a457f98f4ca3611f4f094
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:45.131114 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:45.547015 15326 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:45.551239 15404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:45.573909 15481 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:45.574862 15246 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
3518d7c2b06e475da793e58c284015ac | 127.10.111.62:33399 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0c9b2b8c8da54d1c9f71f932bd56ee30 | 127.10.111.2:46309 | HEALTHY | <none> | 0 | 0
27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | HEALTHY | <none> | 0 | 0
549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | HEALTHY | <none> | 0 | 0
ac8cb44d463a457f98f4ca3611f4f094 | 127.10.111.1:33125 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 05:42:45.889715 10684 log_verifier.cc:126] Checking tablet 44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:45.964453 10684 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:45.965019 10684 tablet_replica.cc:330] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: stopping tablet replica
I20260807 05:42:45.965569 10684 raft_consensus.cc:2229] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:45.966029 10684 raft_consensus.cc:2258] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:46.257839 15326 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:46.273804 15246 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:46.282959 15404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:46.294345 15481 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:46.343173 15217 consensus_queue.cc:579] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: The tablet replica hosted on peer 27c89fae85e14964b2ecba095ce730d2 has failed
W20260807 05:42:46.343700 15217 consensus_peers.cc:487] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 -> Peer 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103): Couldn't send request to peer 27c89fae85e14964b2ecba095ce730d2. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:42:46.376159 15258 consensus_queue.cc:237] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } }
I20260807 05:42:46.400632 15338 raft_consensus.cc:1261] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Refusing update from remote peer ac8cb44d463a457f98f4ca3611f4f094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:42:46.402420 15656 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 05:42:46.421586 15656 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } }
I20260807 05:42:46.426755 15338 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } }
I20260807 05:42:46.434353 15131 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 44e7ee23ffe94dc2b64d9bc396690267 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 05:42:46.440231 15146 catalog_manager.cc:5495] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 reported cstate change: config changed from index -1 to 2, VOTER 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3) evicted. New cstate: current_term: 1 leader_uuid: "ac8cb44d463a457f98f4ca3611f4f094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } health_report { overall_health: HEALTHY } } }
I20260807 05:42:46.535239 15258 consensus_queue.cc:237] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } }
I20260807 05:42:46.537317 15404 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e7ee23ffe94dc2b64d9bc396690267 with delete_type TABLET_DATA_TOMBSTONED (TS 27c89fae85e14964b2ecba095ce730d2 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:57186
I20260807 05:42:46.541836 15665 ts_tablet_manager.cc:1905] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 05:42:46.557492 15665 ts_tablet_manager.cc:1918] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260807 05:42:46.558034 15665 log.cc:1196] T 44e7ee23ffe94dc2b64d9bc396690267 P 27c89fae85e14964b2ecba095ce730d2: Deleting WAL directory at /tmp/dist-test-task6OyI7p/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786081299864053-10684-0/minicluster-data/ts-2-root/wals/44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:46.561054 15129 catalog_manager.cc:4841] TS 27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103): tablet 44e7ee23ffe94dc2b64d9bc396690267 (table test-workload [id=2f62d9b4861a41b89da3fff0fa3b0e99]) successfully deleted
I20260807 05:42:46.571225 15338 raft_consensus.cc:1261] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Refusing update from remote peer ac8cb44d463a457f98f4ca3611f4f094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 05:42:46.573570 15656 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 05:42:46.581128 15613 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 549528e61e034244812eae633e7e3971 (127.10.111.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } } }
I20260807 05:42:46.593309 15338 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 549528e61e034244812eae633e7e3971 (127.10.111.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } } }
I20260807 05:42:46.597738 15131 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 44e7ee23ffe94dc2b64d9bc396690267 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20260807 05:42:46.600503 15218 consensus_peers.cc:487] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 -> Peer 549528e61e034244812eae633e7e3971 (127.10.111.4:38321): Couldn't send request to peer 549528e61e034244812eae633e7e3971. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44e7ee23ffe94dc2b64d9bc396690267. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:42:46.603196 15146 catalog_manager.cc:5495] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 reported cstate change: config changed from index 2 to 3, NON_VOTER 549528e61e034244812eae633e7e3971 (127.10.111.4) added. New cstate: current_term: 1 leader_uuid: "ac8cb44d463a457f98f4ca3611f4f094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
3518d7c2b06e475da793e58c284015ac | 127.10.111.62:33399 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0c9b2b8c8da54d1c9f71f932bd56ee30 | 127.10.111.2:46309 | HEALTHY | <none> | 0 | 0
27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | HEALTHY | <none> | 0 | 0
549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | HEALTHY | <none> | 0 | 0
ac8cb44d463a457f98f4ca3611f4f094 | 127.10.111.1:33125 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 44e7ee23ffe94dc2b64d9bc396690267 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309): RUNNING
27c89fae85e14964b2ecba095ce730d2 (127.10.111.3:34103): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1:33125): RUNNING [LEADER]
All reported replicas are:
A = 0c9b2b8c8da54d1c9f71f932bd56ee30
B = 27c89fae85e14964b2ecba095ce730d2
C = ac8cb44d463a457f98f4ca3611f4f094
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | [config not available] | | |
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:46.667613 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:47.020130 15687 ts_tablet_manager.cc:927] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Initiating tablet copy from peer ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1:33125)
I20260807 05:42:47.022248 15687 tablet_copy_client.cc:316] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: tablet copy: Beginning tablet copy session from remote peer at address 127.10.111.1:33125
I20260807 05:42:47.061491 15268 tablet_copy_service.cc:140] P ac8cb44d463a457f98f4ca3611f4f094: Received BeginTabletCopySession request for tablet 44e7ee23ffe94dc2b64d9bc396690267 from peer 549528e61e034244812eae633e7e3971 ({username='slave'} at 127.0.0.1:57928)
I20260807 05:42:47.062136 15268 tablet_copy_service.cc:161] P ac8cb44d463a457f98f4ca3611f4f094: Beginning new tablet copy session on tablet 44e7ee23ffe94dc2b64d9bc396690267 from peer 549528e61e034244812eae633e7e3971 at {username='slave'} at 127.0.0.1:57928: session id = 549528e61e034244812eae633e7e3971-44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:47.083600 15268 tablet_copy_source_session.cc:205] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:42:47.091925 15687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e7ee23ffe94dc2b64d9bc396690267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:42:47.100652 15481 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:47.100699 15246 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:47.114174 15687 tablet_copy_client.cc:791] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: tablet copy: Starting download of 0 data blocks...
I20260807 05:42:47.115244 15687 tablet_copy_client.cc:655] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: tablet copy: Starting download of 1 WAL segments...
I20260807 05:42:47.120658 15687 tablet_copy_client.cc:531] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:42:47.136081 15687 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Bootstrap starting.
I20260807 05:42:47.158676 15326 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:47.187829 15404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:47.198529 15687 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:42:47.199658 15687 tablet_bootstrap.cc:492] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Bootstrap complete.
I20260807 05:42:47.200519 15687 ts_tablet_manager.cc:1397] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Time spent bootstrapping tablet: real 0.065s user 0.031s sys 0.026s
I20260807 05:42:47.203925 15687 raft_consensus.cc:348] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } }
I20260807 05:42:47.205127 15687 raft_consensus.cc:729] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 549528e61e034244812eae633e7e3971, State: Initialized, Role: LEARNER
I20260807 05:42:47.206248 15687 consensus_queue.cc:260] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: NON_VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: true } }
I20260807 05:42:47.215881 15687 ts_tablet_manager.cc:1428] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: Time spent starting tablet: real 0.015s user 0.005s sys 0.007s
I20260807 05:42:47.219568 15268 tablet_copy_service.cc:342] P ac8cb44d463a457f98f4ca3611f4f094: Request end of tablet copy session 549528e61e034244812eae633e7e3971-44e7ee23ffe94dc2b64d9bc396690267 received from {username='slave'} at 127.0.0.1:57928
I20260807 05:42:47.220120 15268 tablet_copy_service.cc:433] P ac8cb44d463a457f98f4ca3611f4f094: ending tablet copy session 549528e61e034244812eae633e7e3971-44e7ee23ffe94dc2b64d9bc396690267 on tablet 44e7ee23ffe94dc2b64d9bc396690267 with peer 549528e61e034244812eae633e7e3971
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
3518d7c2b06e475da793e58c284015ac | 127.10.111.62:33399 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0c9b2b8c8da54d1c9f71f932bd56ee30 | 127.10.111.2:46309 | HEALTHY | <none> | 0 | 0
27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | HEALTHY | <none> | 0 | 0
549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | HEALTHY | <none> | 0 | 0
ac8cb44d463a457f98f4ca3611f4f094 | 127.10.111.1:33125 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 44e7ee23ffe94dc2b64d9bc396690267 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
0c9b2b8c8da54d1c9f71f932bd56ee30 (127.10.111.2:46309): RUNNING
ac8cb44d463a457f98f4ca3611f4f094 (127.10.111.1:33125): RUNNING [LEADER]
549528e61e034244812eae633e7e3971 (127.10.111.4:38321): missing [NONVOTER]
All reported replicas are:
A = 0c9b2b8c8da54d1c9f71f932bd56ee30
B = ac8cb44d463a457f98f4ca3611f4f094
C = 549528e61e034244812eae633e7e3971
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C~ | | | Yes
A | A B* C~ | 1 | 3 | Yes
B | A B* C~ | 1 | 3 | Yes
C | A B C~ | 1 | 3 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 05:42:47.476536 10684 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 05:42:47.582773 15492 raft_consensus.cc:1203] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3] Dedup: 1.3->[]
I20260807 05:42:48.007134 15326 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:48.026151 15404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:48.030028 15481 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 05:42:48.040271 15246 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 05:42:48.218014 15704 raft_consensus.cc:1050] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: attempting to promote NON_VOTER 549528e61e034244812eae633e7e3971 to VOTER
I20260807 05:42:48.221006 15704 consensus_queue.cc:237] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false } }
I20260807 05:42:48.233580 15492 raft_consensus.cc:1261] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 LEARNER]: Refusing update from remote peer ac8cb44d463a457f98f4ca3611f4f094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 05:42:48.236428 15737 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 05:42:48.242157 15338 raft_consensus.cc:1261] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Refusing update from remote peer ac8cb44d463a457f98f4ca3611f4f094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 05:42:48.245877 15737 consensus_queue.cc:1035] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 05:42:48.264328 15704 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 549528e61e034244812eae633e7e3971 (127.10.111.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false } } }
I20260807 05:42:48.274471 15492 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 549528e61e034244812eae633e7e3971 (127.10.111.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false } } }
I20260807 05:42:48.287794 15146 catalog_manager.cc:5495] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 reported cstate change: config changed from index 3 to 4, 549528e61e034244812eae633e7e3971 (127.10.111.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ac8cb44d463a457f98f4ca3611f4f094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 05:42:48.283732 15338 raft_consensus.cc:2932] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, 549528e61e034244812eae633e7e3971 (127.10.111.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0c9b2b8c8da54d1c9f71f932bd56ee30" member_type: VOTER last_known_addr { host: "127.10.111.2" port: 46309 } } peers { permanent_uuid: "ac8cb44d463a457f98f4ca3611f4f094" member_type: VOTER last_known_addr { host: "127.10.111.1" port: 33125 } } peers { permanent_uuid: "549528e61e034244812eae633e7e3971" member_type: VOTER last_known_addr { host: "127.10.111.4" port: 38321 } attrs { promote: false } } }
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
3518d7c2b06e475da793e58c284015ac | 127.10.111.62:33399 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0c9b2b8c8da54d1c9f71f932bd56ee30 | 127.10.111.2:46309 | HEALTHY | <none> | 0 | 0
27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | HEALTHY | <none> | 0 | 0
549528e61e034244812eae633e7e3971 | 127.10.111.4:38321 | HEALTHY | <none> | 0 | 0
ac8cb44d463a457f98f4ca3611f4f094 | 127.10.111.1:33125 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 27c89fae85e14964b2ecba095ce730d2 | 127.10.111.3:34103 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 05:42:48.384958 10684 log_verifier.cc:126] Checking tablet 44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:48.458555 10684 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 44e7ee23ffe94dc2b64d9bc396690267
I20260807 05:42:48.460811 10684 tablet_server.cc:178] TabletServer@127.10.111.1:0 shutting down...
I20260807 05:42:48.493413 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:48.494048 10684 tablet_replica.cc:330] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094: stopping tablet replica
I20260807 05:42:48.494674 10684 raft_consensus.cc:2229] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:48.495529 10684 raft_consensus.cc:2258] T 44e7ee23ffe94dc2b64d9bc396690267 P ac8cb44d463a457f98f4ca3611f4f094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:49.142496 10684 tablet_server.cc:194] TabletServer@127.10.111.1:0 shutdown complete.
I20260807 05:42:49.153460 10684 tablet_server.cc:178] TabletServer@127.10.111.2:0 shutting down...
I20260807 05:42:49.175710 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:49.176299 10684 tablet_replica.cc:330] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30: stopping tablet replica
I20260807 05:42:49.176937 10684 raft_consensus.cc:2229] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:49.177361 10684 raft_consensus.cc:2258] T 44e7ee23ffe94dc2b64d9bc396690267 P 0c9b2b8c8da54d1c9f71f932bd56ee30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:49.198305 10684 tablet_server.cc:194] TabletServer@127.10.111.2:0 shutdown complete.
I20260807 05:42:49.207312 10684 tablet_server.cc:178] TabletServer@127.10.111.3:0 shutting down...
I20260807 05:42:49.228214 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:49.246148 10684 tablet_server.cc:194] TabletServer@127.10.111.3:0 shutdown complete.
I20260807 05:42:49.255781 10684 tablet_server.cc:178] TabletServer@127.10.111.4:0 shutting down...
I20260807 05:42:49.275650 10684 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 05:42:49.276224 10684 tablet_replica.cc:330] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971: stopping tablet replica
I20260807 05:42:49.276819 10684 raft_consensus.cc:2229] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:42:49.277297 10684 raft_consensus.cc:2258] T 44e7ee23ffe94dc2b64d9bc396690267 P 549528e61e034244812eae633e7e3971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:49.297338 10684 tablet_server.cc:194] TabletServer@127.10.111.4:0 shutdown complete.
I20260807 05:42:49.306892 10684 master.cc:537] Master@127.10.111.62:33399 shutting down...
I20260807 05:42:49.326687 10684 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:42:49.327210 10684 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:42:49.327504 10684 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3518d7c2b06e475da793e58c284015ac: stopping tablet replica
I20260807 05:42:49.346369 10684 master.cc:552] Master@127.10.111.62:33399 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (6078 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (11724 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (69258 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260807 05:42:49.383934 10684 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~12 seconds ago
I20260807 05:42:49.384320 10684 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 10 messages since previous log ~17 seconds ago
I20260807 05:42:49.384483 10684 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 89 messages since previous log ~60 seconds ago
I20260807 05:42:49.384851 10684 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 9 messages since previous log ~16 seconds ago