[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 16:08:19.874176 8080 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.109s user 0.467s sys 0.641s
W20260811 16:08:19.874740 8080 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.109s user 0.467s sys 0.641s
I20260811 16:08:19.879910 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:34383
I20260811 16:08:19.882000 8080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 16:08:19.883370 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:19.900701 8094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:19.902123 8093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:19.998539 8098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:19.998929 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:20.000192 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:20.000430 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:20.000577 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464500000561 us; error 0 us; skew 500 ppm
I20260811 16:08:20.001358 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:20.006928 8080 webserver.cc:460] Webserver started at http://127.7.228.62:33149/ using document root <none> and password file <none>
I20260811 16:08:20.007866 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:20.008086 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:20.008471 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:20.015159 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "5cfc6b8a7fda4b52aca08067fe130c08"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.023502 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 16:08:20.029295 8103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.030767 8080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 16:08:20.031154 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "5cfc6b8a7fda4b52aca08067fe130c08"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.031517 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:20.055240 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:20.056883 8080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 16:08:20.057425 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:20.131489 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:34383
I20260811 16:08:20.131601 8189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:34383 every 8 connection(s)
I20260811 16:08:20.138398 8191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:20.164178 8191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: Bootstrap starting.
I20260811 16:08:20.170938 8191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:20.173101 8191 log.cc:824] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: Log is configured to *not* fsync() on all Append() calls
I20260811 16:08:20.178218 8191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: No bootstrap required, opened a new log
I20260811 16:08:20.202065 8191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER }
I20260811 16:08:20.202802 8191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:20.203022 8191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc6b8a7fda4b52aca08067fe130c08, State: Initialized, Role: FOLLOWER
I20260811 16:08:20.203786 8191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER }
I20260811 16:08:20.204288 8191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:08:20.204536 8191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:08:20.204828 8191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:20.209129 8191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER }
I20260811 16:08:20.209895 8191 leader_election.cc:302] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cfc6b8a7fda4b52aca08067fe130c08; no voters:
I20260811 16:08:20.211684 8191 leader_election.cc:288] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:08:20.212064 8194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:20.214249 8194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 1 LEADER]: Becoming Leader. State: Replica: 5cfc6b8a7fda4b52aca08067fe130c08, State: Running, Role: LEADER
I20260811 16:08:20.215099 8194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER }
I20260811 16:08:20.216248 8191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:20.223305 8199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cfc6b8a7fda4b52aca08067fe130c08. Latest consensus state: current_term: 1 leader_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER } }
I20260811 16:08:20.224130 8199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:20.226847 8198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfc6b8a7fda4b52aca08067fe130c08" member_type: VOTER } }
I20260811 16:08:20.227504 8198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:20.230948 8208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:20.242677 8208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:20.251324 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:20.263046 8208 catalog_manager.cc:1347] Generated new cluster ID: ddc9970220db462bbdc4bcd1d8c30189
I20260811 16:08:20.263327 8208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:20.284494 8208 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:20.286494 8208 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:20.303437 8208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: Generated new TSK 0
I20260811 16:08:20.304615 8208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:20.318542 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:20.327557 8226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:20.330008 8228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:20.333556 8230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:20.334359 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:20.335162 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:20.335362 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:20.335541 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464500335525 us; error 0 us; skew 500 ppm
I20260811 16:08:20.336145 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:20.340502 8080 webserver.cc:460] Webserver started at http://127.7.228.1:33995/ using document root <none> and password file <none>
I20260811 16:08:20.341125 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:20.341338 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:20.341646 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:20.342993 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "99de2747be6047a6a3d05453c7a22359"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.349128 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 16:08:20.353834 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.354791 8080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 16:08:20.355196 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "99de2747be6047a6a3d05453c7a22359"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.355638 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:20.393110 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:20.394905 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:20.397953 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:20.403803 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:20.404176 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.404496 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:20.404732 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.502867 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:43375
I20260811 16:08:20.502933 8338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:43375 every 8 connection(s)
I20260811 16:08:20.514041 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:20.525386 8344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:20.529192 8346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:20.539796 8340 heartbeater.cc:347] Connected to a master server at 127.7.228.62:34383
I20260811 16:08:20.540336 8340 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:20.541546 8340 heartbeater.cc:511] Master 127.7.228.62:34383 requested a full tablet report, sending...
I20260811 16:08:20.545344 8131 ts_manager.cc:194] Registered new tserver with Master: 99de2747be6047a6a3d05453c7a22359 (127.7.228.1:43375)
I20260811 16:08:20.548754 8131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55594
W20260811 16:08:20.556761 8348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:20.557632 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:20.558537 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:20.558779 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:20.558931 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464500558916 us; error 0 us; skew 500 ppm
I20260811 16:08:20.559437 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:20.562345 8080 webserver.cc:460] Webserver started at http://127.7.228.2:44497/ using document root <none> and password file <none>
I20260811 16:08:20.562871 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:20.563067 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:20.563318 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:20.564456 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "a5e7c79a9da843ffad32ae03185fbadc"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.570112 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 16:08:20.574687 8355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.575537 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 16:08:20.575824 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "a5e7c79a9da843ffad32ae03185fbadc"
format_stamp: "Formatted at 2026-08-11 16:08:20 on dist-test-slave-ncp9"
I20260811 16:08:20.576112 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:20.594189 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:20.595489 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:20.597996 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:20.601225 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:20.601425 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.601679 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:20.601817 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:20.653785 8340 heartbeater.cc:503] Master 127.7.228.62:34383 was elected leader, sending a full tablet report...
I20260811 16:08:20.703244 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:35767
I20260811 16:08:20.703408 8454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:35767 every 8 connection(s)
I20260811 16:08:20.725126 8455 heartbeater.cc:347] Connected to a master server at 127.7.228.62:34383
I20260811 16:08:20.725522 8455 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:20.726328 8455 heartbeater.cc:511] Master 127.7.228.62:34383 requested a full tablet report, sending...
I20260811 16:08:20.728485 8131 ts_manager.cc:194] Registered new tserver with Master: a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2:35767)
I20260811 16:08:20.728845 8080 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.020019139s
I20260811 16:08:20.730773 8131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55604
I20260811 16:08:20.764671 8131 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260811 16:08:20.787858 8131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260811 16:08:20.834807 8455 heartbeater.cc:503] Master 127.7.228.62:34383 was elected leader, sending a full tablet report...
I20260811 16:08:20.855144 8403 tablet_service.cc:1467] Processing CreateTablet for tablet 3c8f2dc2e855475a993c4a01365e333b (DEFAULT_TABLE table=test-table [id=0467938804004dce819da963f2892a39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:20.855288 8404 tablet_service.cc:1467] Processing CreateTablet for tablet ddd58980068441a0b59b30b13acde434 (DEFAULT_TABLE table=test-table [id=0467938804004dce819da963f2892a39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:20.856940 8403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c8f2dc2e855475a993c4a01365e333b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:20.857087 8278 tablet_service.cc:1467] Processing CreateTablet for tablet 3c8f2dc2e855475a993c4a01365e333b (DEFAULT_TABLE table=test-table [id=0467938804004dce819da963f2892a39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:20.857669 8404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd58980068441a0b59b30b13acde434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:20.858560 8278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c8f2dc2e855475a993c4a01365e333b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:20.862082 8279 tablet_service.cc:1467] Processing CreateTablet for tablet ddd58980068441a0b59b30b13acde434 (DEFAULT_TABLE table=test-table [id=0467938804004dce819da963f2892a39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:20.863571 8279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd58980068441a0b59b30b13acde434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:20.883628 8474 tablet_bootstrap.cc:492] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: Bootstrap starting.
I20260811 16:08:20.886634 8477 tablet_bootstrap.cc:492] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: Bootstrap starting.
I20260811 16:08:20.890645 8474 tablet_bootstrap.cc:654] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:20.893927 8477 tablet_bootstrap.cc:654] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:20.900171 8477 tablet_bootstrap.cc:492] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: No bootstrap required, opened a new log
I20260811 16:08:20.900171 8474 tablet_bootstrap.cc:492] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: No bootstrap required, opened a new log
I20260811 16:08:20.900760 8474 ts_tablet_manager.cc:1397] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260811 16:08:20.900764 8477 ts_tablet_manager.cc:1397] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260811 16:08:20.903978 8477 raft_consensus.cc:348] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:20.903893 8474 raft_consensus.cc:348] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:20.904743 8477 raft_consensus.cc:729] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99de2747be6047a6a3d05453c7a22359, State: Initialized, Role: FOLLOWER
I20260811 16:08:20.904846 8474 raft_consensus.cc:729] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Initialized, Role: FOLLOWER
I20260811 16:08:20.905508 8474 consensus_queue.cc:260] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:20.905509 8477 consensus_queue.cc:260] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:20.910957 8474 ts_tablet_manager.cc:1428] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260811 16:08:20.912667 8474 tablet_bootstrap.cc:492] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: Bootstrap starting.
I20260811 16:08:20.915999 8477 ts_tablet_manager.cc:1428] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: Time spent starting tablet: real 0.015s user 0.006s sys 0.006s
I20260811 16:08:20.918607 8477 tablet_bootstrap.cc:492] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: Bootstrap starting.
I20260811 16:08:20.919203 8474 tablet_bootstrap.cc:654] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:20.925004 8477 tablet_bootstrap.cc:654] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:20.932530 8474 tablet_bootstrap.cc:492] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: No bootstrap required, opened a new log
I20260811 16:08:20.933169 8474 ts_tablet_manager.cc:1397] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.005s
I20260811 16:08:20.936316 8474 raft_consensus.cc:348] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:20.937127 8474 raft_consensus.cc:729] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Initialized, Role: FOLLOWER
I20260811 16:08:20.937536 8477 tablet_bootstrap.cc:492] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: No bootstrap required, opened a new log
I20260811 16:08:20.938098 8477 ts_tablet_manager.cc:1397] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.008s
I20260811 16:08:20.937764 8474 consensus_queue.cc:260] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:20.941048 8477 raft_consensus.cc:348] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:20.941939 8477 raft_consensus.cc:729] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99de2747be6047a6a3d05453c7a22359, State: Initialized, Role: FOLLOWER
I20260811 16:08:20.942647 8477 consensus_queue.cc:260] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:20.945128 8477 ts_tablet_manager.cc:1428] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 16:08:20.948596 8474 ts_tablet_manager.cc:1428] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
W20260811 16:08:20.961874 8456 tablet.cc:2369] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 16:08:20.980757 8080 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260811 16:08:20.981151 8080 raft_consensus.cc:412] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Emulating election...
I20260811 16:08:20.981428 8080 raft_consensus.cc:3037] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:20.987406 8080 raft_consensus.cc:686] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Becoming Leader. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:20.988240 8080 consensus_queue.cc:237] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:20.992767 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:21.000339 8129 catalog_manager.cc:5495] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc reported cstate change: term changed from 0 to 1, leader changed from <none> to a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2). New cstate: current_term: 1 leader_uuid: "a5e7c79a9da843ffad32ae03185fbadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:21.026504 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:21.130932 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:21.337705 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:21.425705 8301 raft_consensus.cc:3037] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:21.438673 8491 consensus_queue.cc:1035] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 16:08:21.643344 8080 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260811 16:08:21.643627 8080 raft_consensus.cc:412] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Emulating election...
I20260811 16:08:21.643843 8080 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:21.647992 8080 raft_consensus.cc:686] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Becoming Leader. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:21.648780 8080 consensus_queue.cc:237] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:21.652591 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:21.655160 8129 catalog_manager.cc:5495] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc reported cstate change: term changed from 0 to 1, leader changed from <none> to a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2). New cstate: current_term: 1 leader_uuid: "a5e7c79a9da843ffad32ae03185fbadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:21.656117 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:21.760850 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:21.964964 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260811 16:08:22.219811 8301 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:22.225549 8491 consensus_queue.cc:1035] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 16:08:22.469848 8080 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260811 16:08:22.470103 8080 raft_consensus.cc:412] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Emulating election...
I20260811 16:08:22.470350 8080 raft_consensus.cc:3032] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:22.470533 8080 raft_consensus.cc:729] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:22.471020 8080 consensus_queue.cc:260] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:22.471873 8080 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:22.475816 8080 raft_consensus.cc:686] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 2 LEADER]: Becoming Leader. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:22.476574 8080 consensus_queue.cc:237] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:22.478186 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:22.481563 8131 catalog_manager.cc:5495] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "a5e7c79a9da843ffad32ae03185fbadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:22.482004 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260811 16:08:22.586127 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260811 16:08:22.790298 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260811 16:08:22.931766 8301 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:22.937156 8301 raft_consensus.cc:1261] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 2 FOLLOWER]: Refusing update from remote peer a5e7c79a9da843ffad32ae03185fbadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 16:08:22.938365 8516 consensus_queue.cc:1035] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:08:23.094739 8080 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260811 16:08:23.095057 8080 raft_consensus.cc:412] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 2 FOLLOWER]: Emulating election...
I20260811 16:08:23.095319 8080 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:23.099270 8080 raft_consensus.cc:686] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 3 LEADER]: Becoming Leader. State: Replica: 99de2747be6047a6a3d05453c7a22359, State: Running, Role: LEADER
I20260811 16:08:23.099812 8080 consensus_queue.cc:237] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:23.101418 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:23.105345 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260811 16:08:23.105111 8131 catalog_manager.cc:5495] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 reported cstate change: term changed from 2 to 3, leader changed from a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2) to 99de2747be6047a6a3d05453c7a22359 (127.7.228.1). New cstate: current_term: 3 leader_uuid: "99de2747be6047a6a3d05453c7a22359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:23.209640 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260811 16:08:23.405310 8301 raft_consensus.cc:1226] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 3 LEADER]: Rejecting Update request from peer a5e7c79a9da843ffad32ae03185fbadc for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:23.406679 8516 consensus_queue.cc:1046] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:23.407456 8516 consensus_queue.cc:1225] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Peer 99de2747be6047a6a3d05453c7a22359 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260811 16:08:23.407874 8523 raft_consensus.cc:3032] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:23.408246 8523 raft_consensus.cc:729] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:23.408881 8523 consensus_queue.cc:260] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:23.409667 8523 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:23.417166 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260811 16:08:23.575091 8417 raft_consensus.cc:1261] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 3 FOLLOWER]: Refusing update from remote peer 99de2747be6047a6a3d05453c7a22359: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 16:08:23.576254 8522 consensus_queue.cc:1035] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:23.921819 8080 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260811 16:08:23.922124 8080 raft_consensus.cc:412] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 1 FOLLOWER]: Emulating election...
I20260811 16:08:23.922385 8080 raft_consensus.cc:3037] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:23.926584 8080 raft_consensus.cc:686] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 2 LEADER]: Becoming Leader. State: Replica: 99de2747be6047a6a3d05453c7a22359, State: Running, Role: LEADER
I20260811 16:08:23.927227 8080 consensus_queue.cc:237] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:23.929373 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:23.933405 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260811 16:08:23.933172 8131 catalog_manager.cc:5495] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 reported cstate change: term changed from 1 to 2, leader changed from a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2) to 99de2747be6047a6a3d05453c7a22359 (127.7.228.1). New cstate: current_term: 2 leader_uuid: "99de2747be6047a6a3d05453c7a22359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:24.037211 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260811 16:08:24.154191 8301 raft_consensus.cc:1226] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 2 LEADER]: Rejecting Update request from peer a5e7c79a9da843ffad32ae03185fbadc for earlier term 1. Current term is 2. Ops: []
I20260811 16:08:24.155467 8550 consensus_queue.cc:1046] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:08:24.156478 8550 consensus_queue.cc:1225] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Peer 99de2747be6047a6a3d05453c7a22359 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260811 16:08:24.156920 8552 raft_consensus.cc:3032] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:24.157215 8552 raft_consensus.cc:729] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:24.157711 8552 consensus_queue.cc:260] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } } peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } }
I20260811 16:08:24.158542 8552 raft_consensus.cc:3037] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:24.241462 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260811 16:08:24.457056 8417 raft_consensus.cc:1261] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 2 FOLLOWER]: Refusing update from remote peer 99de2747be6047a6a3d05453c7a22359: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 16:08:24.458459 8522 consensus_queue.cc:1035] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:08:24.545713 8080 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260811 16:08:24.545950 8080 raft_consensus.cc:412] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 3 FOLLOWER]: Emulating election...
I20260811 16:08:24.546231 8080 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:24.551295 8080 raft_consensus.cc:686] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 4 LEADER]: Becoming Leader. State: Replica: a5e7c79a9da843ffad32ae03185fbadc, State: Running, Role: LEADER
I20260811 16:08:24.551925 8080 consensus_queue.cc:237] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:24.553284 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:24.556735 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260811 16:08:24.556622 8129 catalog_manager.cc:5495] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc reported cstate change: term changed from 3 to 4, leader changed from 99de2747be6047a6a3d05453c7a22359 (127.7.228.1) to a5e7c79a9da843ffad32ae03185fbadc (127.7.228.2). New cstate: current_term: 4 leader_uuid: "a5e7c79a9da843ffad32ae03185fbadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:24.601711 8417 raft_consensus.cc:1226] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 4 LEADER]: Rejecting Update request from peer 99de2747be6047a6a3d05453c7a22359 for earlier term 3. Current term is 4. Ops: []
I20260811 16:08:24.603083 8522 consensus_queue.cc:1046] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:24.603653 8522 consensus_queue.cc:1225] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [LEADER]: Peer a5e7c79a9da843ffad32ae03185fbadc log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260811 16:08:24.604017 8554 raft_consensus.cc:3032] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 16:08:24.604297 8554 raft_consensus.cc:729] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99de2747be6047a6a3d05453c7a22359, State: Running, Role: LEADER
I20260811 16:08:24.604791 8554 consensus_queue.cc:260] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 } } peers { permanent_uuid: "a5e7c79a9da843ffad32ae03185fbadc" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 35767 } }
I20260811 16:08:24.605566 8554 raft_consensus.cc:3037] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:24.660619 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260811 16:08:24.864526 8080 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260811 16:08:24.975534 8301 raft_consensus.cc:1261] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 4 FOLLOWER]: Refusing update from remote peer a5e7c79a9da843ffad32ae03185fbadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260811 16:08:24.976832 8552 consensus_queue.cc:1035] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "99de2747be6047a6a3d05453c7a22359" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:25.371275 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:08:25.391561 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:25.392184 8080 tablet_replica.cc:330] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359: stopping tablet replica
I20260811 16:08:25.392781 8080 raft_consensus.cc:2229] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260811 16:08:25.393287 8080 raft_consensus.cc:2258] T 3c8f2dc2e855475a993c4a01365e333b P 99de2747be6047a6a3d05453c7a22359 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:25.395753 8080 tablet_replica.cc:330] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359: stopping tablet replica
I20260811 16:08:25.396222 8080 raft_consensus.cc:2229] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 2 LEADER]: Raft consensus shutting down.
I20260811 16:08:25.396770 8080 raft_consensus.cc:2258] T ddd58980068441a0b59b30b13acde434 P 99de2747be6047a6a3d05453c7a22359 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260811 16:08:25.493798 8358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.228.1:43375: connect: Connection refused (error 111)
I20260811 16:08:25.499981 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
W20260811 16:08:25.502942 8358 consensus_peers.cc:487] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc -> Peer 99de2747be6047a6a3d05453c7a22359 (127.7.228.1:43375): Couldn't send request to peer 99de2747be6047a6a3d05453c7a22359. Status: Network error: Client connection negotiation failed: client connection to 127.7.228.1:43375: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 16:08:25.511029 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:08:25.526472 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:25.527069 8080 tablet_replica.cc:330] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc: stopping tablet replica
I20260811 16:08:25.527602 8080 raft_consensus.cc:2229] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 16:08:25.528007 8080 raft_consensus.cc:2258] T ddd58980068441a0b59b30b13acde434 P a5e7c79a9da843ffad32ae03185fbadc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:25.529523 8080 tablet_replica.cc:330] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc: stopping tablet replica
I20260811 16:08:25.529943 8080 raft_consensus.cc:2229] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 4 LEADER]: Raft consensus shutting down.
I20260811 16:08:25.530511 8080 raft_consensus.cc:2258] T 3c8f2dc2e855475a993c4a01365e333b P a5e7c79a9da843ffad32ae03185fbadc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:25.550037 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:08:25.558921 8080 master.cc:537] Master@127.7.228.62:34383 shutting down...
I20260811 16:08:25.831200 8224 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260811 16:08:25.975019 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:25.975536 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:25.975899 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5cfc6b8a7fda4b52aca08067fe130c08: stopping tablet replica
I20260811 16:08:25.993278 8080 master.cc:552] Master@127.7.228.62:34383 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7286 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260811 16:08:26.022014 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:45941
I20260811 16:08:26.023272 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.028245 8583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.029753 8584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.030675 8586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.031257 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:26.031981 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.032194 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.032315 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506032306 us; error 0 us; skew 500 ppm
I20260811 16:08:26.032788 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.038828 8080 webserver.cc:460] Webserver started at http://127.7.228.62:40805/ using document root <none> and password file <none>
I20260811 16:08:26.039318 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.039484 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.039710 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.040788 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "d34c36cd878948539c3857117ecc74ce"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.045682 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 16:08:26.049300 8593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.050046 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:08:26.050305 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "d34c36cd878948539c3857117ecc74ce"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.050551 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.069132 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.070129 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.104866 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:45941
I20260811 16:08:26.104990 8662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:45941 every 8 connection(s)
I20260811 16:08:26.108654 8664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:26.117995 8664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce: Bootstrap starting.
I20260811 16:08:26.121976 8664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:26.125893 8664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce: No bootstrap required, opened a new log
I20260811 16:08:26.127660 8664 raft_consensus.cc:348] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER }
I20260811 16:08:26.128044 8664 raft_consensus.cc:374] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:26.128281 8664 raft_consensus.cc:729] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34c36cd878948539c3857117ecc74ce, State: Initialized, Role: FOLLOWER
I20260811 16:08:26.128808 8664 consensus_queue.cc:260] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER }
I20260811 16:08:26.129282 8664 raft_consensus.cc:388] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:08:26.129513 8664 raft_consensus.cc:482] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:08:26.129781 8664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:26.133793 8664 raft_consensus.cc:504] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER }
I20260811 16:08:26.134325 8664 leader_election.cc:302] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d34c36cd878948539c3857117ecc74ce; no voters:
I20260811 16:08:26.135432 8664 leader_election.cc:288] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:08:26.135852 8668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:26.137127 8668 raft_consensus.cc:686] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 1 LEADER]: Becoming Leader. State: Replica: d34c36cd878948539c3857117ecc74ce, State: Running, Role: LEADER
I20260811 16:08:26.137771 8668 consensus_queue.cc:237] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER }
I20260811 16:08:26.138123 8664 sys_catalog.cc:564] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:26.140520 8670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d34c36cd878948539c3857117ecc74ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER } }
I20260811 16:08:26.141346 8670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:26.140545 8671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader d34c36cd878948539c3857117ecc74ce. Latest consensus state: current_term: 1 leader_uuid: "d34c36cd878948539c3857117ecc74ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34c36cd878948539c3857117ecc74ce" member_type: VOTER } }
I20260811 16:08:26.142585 8671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:26.144304 8676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:26.149266 8676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:26.152252 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:26.157788 8676 catalog_manager.cc:1347] Generated new cluster ID: 41f7b884df6c451a9af03a9f8298ac7b
I20260811 16:08:26.158104 8676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:26.188297 8676 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:26.189946 8676 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:26.200254 8676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce: Generated new TSK 0
I20260811 16:08:26.200922 8676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:26.219558 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.225353 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.226294 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.227738 8698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.228499 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:26.229352 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.229533 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.229692 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506229670 us; error 0 us; skew 500 ppm
I20260811 16:08:26.230187 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.232612 8080 webserver.cc:460] Webserver started at http://127.7.228.1:41063/ using document root <none> and password file <none>
I20260811 16:08:26.233151 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.233325 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.233572 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.234651 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "e71e49e20ba641b48c0db43423a8452f"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.239246 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:08:26.242573 8707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.243348 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 16:08:26.243625 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "e71e49e20ba641b48c0db43423a8452f"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.243896 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.257658 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.258766 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.260203 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:26.262482 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:26.262686 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.262907 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:26.263048 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.301409 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:34077
I20260811 16:08:26.301487 8800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:34077 every 8 connection(s)
I20260811 16:08:26.306083 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.312664 8806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.314662 8809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.318048 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:26.318738 8812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.319770 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.320043 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.320246 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506320227 us; error 0 us; skew 500 ppm
I20260811 16:08:26.320851 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.321578 8801 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45941
I20260811 16:08:26.322000 8801 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:26.322825 8801 heartbeater.cc:511] Master 127.7.228.62:45941 requested a full tablet report, sending...
I20260811 16:08:26.323819 8080 webserver.cc:460] Webserver started at http://127.7.228.2:41321/ using document root <none> and password file <none>
I20260811 16:08:26.324402 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.324595 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.324867 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.325181 8613 ts_manager.cc:194] Registered new tserver with Master: e71e49e20ba641b48c0db43423a8452f (127.7.228.1:34077)
I20260811 16:08:26.326082 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "fbec33e370b34c7ea09d8e4f427c6c2e"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.327136 8613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38596
I20260811 16:08:26.331272 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 16:08:26.334848 8818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.335721 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 16:08:26.335991 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "fbec33e370b34c7ea09d8e4f427c6c2e"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.336251 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.355669 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.356997 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.358485 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:26.360713 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:26.360929 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.361141 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:26.361263 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.399176 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:36763
I20260811 16:08:26.399255 8917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:36763 every 8 connection(s)
I20260811 16:08:26.414348 8918 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45941
I20260811 16:08:26.414727 8918 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:26.415408 8918 heartbeater.cc:511] Master 127.7.228.62:45941 requested a full tablet report, sending...
I20260811 16:08:26.417289 8613 ts_manager.cc:194] Registered new tserver with Master: fbec33e370b34c7ea09d8e4f427c6c2e (127.7.228.2:36763)
I20260811 16:08:26.417766 8080 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015655461s
I20260811 16:08:26.419359 8613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38610
I20260811 16:08:26.430195 8801 heartbeater.cc:503] Master 127.7.228.62:45941 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260811 16:08:26.442355 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:08:26.461187 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:26.476661 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:08:26.483433 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:08:26.498558 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:26.514446 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:08:26.521553 8080 master.cc:537] Master@127.7.228.62:45941 shutting down...
I20260811 16:08:26.535027 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:26.535555 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:26.535915 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P d34c36cd878948539c3857117ecc74ce: stopping tablet replica
I20260811 16:08:26.555045 8080 master.cc:552] Master@127.7.228.62:45941 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (555 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260811 16:08:26.576953 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:39405
I20260811 16:08:26.577968 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.582602 8937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.583784 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.586400 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.586455 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:26.587218 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.587389 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.587545 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506587533 us; error 0 us; skew 500 ppm
I20260811 16:08:26.588064 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.590584 8080 webserver.cc:460] Webserver started at http://127.7.228.62:38071/ using document root <none> and password file <none>
I20260811 16:08:26.591049 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.591217 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.591454 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.592496 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "21a773963387432b9f8e12ac2f7e8f99"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.596858 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 16:08:26.600356 8947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.601121 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 16:08:26.601392 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "21a773963387432b9f8e12ac2f7e8f99"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.601665 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.620599 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.621676 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.657301 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:39405
I20260811 16:08:26.657415 9027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:39405 every 8 connection(s)
I20260811 16:08:26.661226 9028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:26.670957 9028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99: Bootstrap starting.
I20260811 16:08:26.675410 9028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:26.679487 9028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99: No bootstrap required, opened a new log
I20260811 16:08:26.681348 9028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER }
I20260811 16:08:26.681752 9028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:26.681929 9028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a773963387432b9f8e12ac2f7e8f99, State: Initialized, Role: FOLLOWER
I20260811 16:08:26.682425 9028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER }
I20260811 16:08:26.682858 9028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:08:26.683060 9028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:08:26.683267 9028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:26.687376 9028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER }
I20260811 16:08:26.687904 9028 leader_election.cc:302] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a773963387432b9f8e12ac2f7e8f99; no voters:
I20260811 16:08:26.689247 9028 leader_election.cc:288] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:08:26.689471 9031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:26.690833 9031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 1 LEADER]: Becoming Leader. State: Replica: 21a773963387432b9f8e12ac2f7e8f99, State: Running, Role: LEADER
I20260811 16:08:26.691660 9031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER }
I20260811 16:08:26.692008 9028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:26.694759 9033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a773963387432b9f8e12ac2f7e8f99. Latest consensus state: current_term: 1 leader_uuid: "21a773963387432b9f8e12ac2f7e8f99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER } }
I20260811 16:08:26.694715 9032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a773963387432b9f8e12ac2f7e8f99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a773963387432b9f8e12ac2f7e8f99" member_type: VOTER } }
I20260811 16:08:26.695490 9033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:26.695626 9032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:26.698196 9037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:26.705408 9037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:26.707221 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:26.715225 9037 catalog_manager.cc:1347] Generated new cluster ID: 6904c596f4564c1484501abce5d33729
I20260811 16:08:26.715561 9037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:26.731159 9037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:26.732811 9037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:26.748503 9037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99: Generated new TSK 0
I20260811 16:08:26.749248 9037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:26.773950 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.779597 9056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.780474 9057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.783353 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:26.783650 9060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.784581 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.784770 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.784883 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506784874 us; error 0 us; skew 500 ppm
I20260811 16:08:26.785386 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.787828 8080 webserver.cc:460] Webserver started at http://127.7.228.1:41015/ using document root <none> and password file <none>
I20260811 16:08:26.788286 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.788437 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.788650 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.789752 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "0eaada8d2c654d82aaa4f51af611b115"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.794562 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 16:08:26.798370 9072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.799265 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:08:26.799564 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "0eaada8d2c654d82aaa4f51af611b115"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.799844 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.811976 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.813210 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.814673 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:26.816958 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:26.817167 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.817394 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:26.817536 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.854908 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:44799
I20260811 16:08:26.854960 9155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:44799 every 8 connection(s)
I20260811 16:08:26.859313 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:26.866248 9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.867974 9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:26.875392 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:26.876262 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:26.877151 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:26.877416 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:26.877619 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464506877604 us; error 0 us; skew 500 ppm
I20260811 16:08:26.878294 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:26.879491 9156 heartbeater.cc:347] Connected to a master server at 127.7.228.62:39405
I20260811 16:08:26.879884 9156 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:26.880514 9156 heartbeater.cc:511] Master 127.7.228.62:39405 requested a full tablet report, sending...
I20260811 16:08:26.881027 8080 webserver.cc:460] Webserver started at http://127.7.228.2:35257/ using document root <none> and password file <none>
I20260811 16:08:26.881608 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:26.881834 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:26.882094 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:26.882520 8971 ts_manager.cc:194] Registered new tserver with Master: 0eaada8d2c654d82aaa4f51af611b115 (127.7.228.1:44799)
I20260811 16:08:26.883255 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "ccdd6b82a17f40849e99f38022a1761d"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.884359 8971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45632
I20260811 16:08:26.887923 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:08:26.891661 9172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.892473 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:08:26.892819 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "ccdd6b82a17f40849e99f38022a1761d"
format_stamp: "Formatted at 2026-08-11 16:08:26 on dist-test-slave-ncp9"
I20260811 16:08:26.893208 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:26.916620 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:26.917663 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:26.919100 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:26.921370 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:26.921571 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.921797 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:26.921938 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:26.959261 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:34779
I20260811 16:08:26.959326 9263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:34779 every 8 connection(s)
I20260811 16:08:26.974117 9265 heartbeater.cc:347] Connected to a master server at 127.7.228.62:39405
I20260811 16:08:26.974489 9265 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:26.975112 9265 heartbeater.cc:511] Master 127.7.228.62:39405 requested a full tablet report, sending...
I20260811 16:08:26.977052 8971 ts_manager.cc:194] Registered new tserver with Master: ccdd6b82a17f40849e99f38022a1761d (127.7.228.2:34779)
I20260811 16:08:26.977648 8080 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015190675s
I20260811 16:08:26.978615 8971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45642
I20260811 16:08:26.987247 9156 heartbeater.cc:503] Master 127.7.228.62:39405 was elected leader, sending a full tablet report...
I20260811 16:08:27.002197 8971 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 16:08:27.003652 8971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260811 16:08:27.037658 9208 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ae5fce9ac4d7995a5d9a567409fc1 (DEFAULT_TABLE table=test-table [id=40e8bae9700f4b488b91726d98747aa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:27.038954 9208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ae5fce9ac4d7995a5d9a567409fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:27.039505 9109 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ae5fce9ac4d7995a5d9a567409fc1 (DEFAULT_TABLE table=test-table [id=40e8bae9700f4b488b91726d98747aa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:27.040467 9109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ae5fce9ac4d7995a5d9a567409fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:27.051261 9281 tablet_bootstrap.cc:492] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: Bootstrap starting.
I20260811 16:08:27.053114 9283 tablet_bootstrap.cc:492] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: Bootstrap starting.
I20260811 16:08:27.055879 9281 tablet_bootstrap.cc:654] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:27.057777 9283 tablet_bootstrap.cc:654] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:27.060266 9281 tablet_bootstrap.cc:492] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: No bootstrap required, opened a new log
I20260811 16:08:27.060681 9281 ts_tablet_manager.cc:1397] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 16:08:27.062543 9281 raft_consensus.cc:348] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.062825 9283 tablet_bootstrap.cc:492] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: No bootstrap required, opened a new log
I20260811 16:08:27.063050 9281 raft_consensus.cc:729] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdd6b82a17f40849e99f38022a1761d, State: Initialized, Role: FOLLOWER
I20260811 16:08:27.063251 9283 ts_tablet_manager.cc:1397] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260811 16:08:27.063551 9281 consensus_queue.cc:260] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.065414 9283 raft_consensus.cc:348] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.065788 9265 heartbeater.cc:503] Master 127.7.228.62:39405 was elected leader, sending a full tablet report...
I20260811 16:08:27.066021 9281 ts_tablet_manager.cc:1428] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 16:08:27.066011 9283 raft_consensus.cc:729] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Initialized, Role: FOLLOWER
I20260811 16:08:27.066726 9283 consensus_queue.cc:260] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.070237 9283 ts_tablet_manager.cc:1428] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260811 16:08:27.100313 8080 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260811 16:08:27.100533 8080 raft_consensus.cc:412] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 0 FOLLOWER]: Emulating election...
I20260811 16:08:27.100790 8080 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:27.105134 8080 raft_consensus.cc:686] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 1 LEADER]: Becoming Leader. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Running, Role: LEADER
W20260811 16:08:27.105517 9156 heartbeater.cc:644] Failed to heartbeat to 127.7.228.62:39405 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260811 16:08:27.105805 8080 consensus_queue.cc:237] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.107998 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:27.129865 8080 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260811 16:08:27.234367 8080 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260811 16:08:27.438619 8080 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260811 16:08:27.510425 9229 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:27.516399 9296 consensus_queue.cc:1035] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:27.744261 8080 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260811 16:08:27.744488 8080 raft_consensus.cc:412] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 1 LEADER]: Emulating election...
I20260811 16:08:27.744697 8080 raft_consensus.cc:3032] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:27.744843 8080 raft_consensus.cc:729] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Running, Role: LEADER
I20260811 16:08:27.745254 8080 consensus_queue.cc:260] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.745873 8080 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:27.749827 8080 raft_consensus.cc:686] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 2 LEADER]: Becoming Leader. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Running, Role: LEADER
I20260811 16:08:27.750401 8080 consensus_queue.cc:237] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:27.751834 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:27.755681 8080 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260811 16:08:27.859751 8080 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260811 16:08:28.064350 8080 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260811 16:08:28.348130 9229 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:28.353391 9229 raft_consensus.cc:1261] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 2 FOLLOWER]: Refusing update from remote peer 0eaada8d2c654d82aaa4f51af611b115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 16:08:28.354918 9305 consensus_queue.cc:1035] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:08:28.369303 8080 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260811 16:08:28.369530 8080 raft_consensus.cc:412] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 2 LEADER]: Emulating election...
I20260811 16:08:28.369746 8080 raft_consensus.cc:3032] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:28.369933 8080 raft_consensus.cc:729] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Running, Role: LEADER
I20260811 16:08:28.370491 8080 consensus_queue.cc:260] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:28.371220 8080 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:28.375543 8080 raft_consensus.cc:686] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 3 LEADER]: Becoming Leader. State: Replica: 0eaada8d2c654d82aaa4f51af611b115, State: Running, Role: LEADER
I20260811 16:08:28.376129 8080 consensus_queue.cc:237] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 } } peers { permanent_uuid: "0eaada8d2c654d82aaa4f51af611b115" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 44799 } }
I20260811 16:08:28.377511 8080 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260811 16:08:28.381242 8080 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260811 16:08:28.485376 8080 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260811 16:08:28.689510 8080 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260811 16:08:28.959040 9229 raft_consensus.cc:3037] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:28.965467 9229 raft_consensus.cc:1261] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 3 FOLLOWER]: Refusing update from remote peer 0eaada8d2c654d82aaa4f51af611b115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 16:08:28.967374 9316 consensus_queue.cc:1035] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdd6b82a17f40849e99f38022a1761d" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260811 16:08:28.999229 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:08:29.024936 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:29.025724 8080 tablet_replica.cc:330] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115: stopping tablet replica
I20260811 16:08:29.026276 8080 raft_consensus.cc:2229] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 3 LEADER]: Raft consensus shutting down.
I20260811 16:08:29.026912 8080 raft_consensus.cc:2258] T cb9ae5fce9ac4d7995a5d9a567409fc1 P 0eaada8d2c654d82aaa4f51af611b115 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:29.036538 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:08:29.046271 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:08:29.067070 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:29.068120 8080 tablet_replica.cc:330] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d: stopping tablet replica
I20260811 16:08:29.068795 8080 raft_consensus.cc:2229] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 16:08:29.069316 8080 raft_consensus.cc:2258] T cb9ae5fce9ac4d7995a5d9a567409fc1 P ccdd6b82a17f40849e99f38022a1761d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:29.090087 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:08:29.101193 8080 master.cc:537] Master@127.7.228.62:39405 shutting down...
I20260811 16:08:29.119863 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:29.120491 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:29.120769 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 21a773963387432b9f8e12ac2f7e8f99: stopping tablet replica
I20260811 16:08:29.138945 8080 master.cc:552] Master@127.7.228.62:39405 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2593 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10434 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260811 16:08:29.170603 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:35801
I20260811 16:08:29.171700 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:29.177568 9329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.178369 9330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.180318 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:29.180691 9334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.181537 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:29.181736 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:29.181847 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464509181838 us; error 0 us; skew 500 ppm
I20260811 16:08:29.182328 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:29.185156 8080 webserver.cc:460] Webserver started at http://127.7.228.62:45377/ using document root <none> and password file <none>
I20260811 16:08:29.185645 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:29.185806 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:29.186051 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:29.187273 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "cd56acffb9024d27b1b2b777fc49960b"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.192306 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 16:08:29.195887 9342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.196719 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:08:29.197041 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "cd56acffb9024d27b1b2b777fc49960b"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.197319 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:29.221869 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:29.223112 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:29.262518 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:35801
I20260811 16:08:29.262609 9422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:35801 every 8 connection(s)
I20260811 16:08:29.266652 9424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:29.277961 9424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b: Bootstrap starting.
I20260811 16:08:29.282483 9424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:29.286764 9424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b: No bootstrap required, opened a new log
I20260811 16:08:29.288753 9424 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER }
I20260811 16:08:29.289232 9424 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:29.289410 9424 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd56acffb9024d27b1b2b777fc49960b, State: Initialized, Role: FOLLOWER
I20260811 16:08:29.289889 9424 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER }
I20260811 16:08:29.290308 9424 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:08:29.290489 9424 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:08:29.290701 9424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:29.295037 9424 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER }
I20260811 16:08:29.295580 9424 leader_election.cc:302] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd56acffb9024d27b1b2b777fc49960b; no voters:
I20260811 16:08:29.296713 9424 leader_election.cc:288] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:08:29.297183 9428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:29.298589 9428 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 1 LEADER]: Becoming Leader. State: Replica: cd56acffb9024d27b1b2b777fc49960b, State: Running, Role: LEADER
I20260811 16:08:29.299263 9428 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER }
I20260811 16:08:29.299803 9424 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:29.302179 9430 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd56acffb9024d27b1b2b777fc49960b. Latest consensus state: current_term: 1 leader_uuid: "cd56acffb9024d27b1b2b777fc49960b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER } }
I20260811 16:08:29.302233 9429 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd56acffb9024d27b1b2b777fc49960b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd56acffb9024d27b1b2b777fc49960b" member_type: VOTER } }
I20260811 16:08:29.303023 9430 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:29.303081 9429 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:29.307471 9440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:29.314761 9440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:29.315865 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:29.323751 9440 catalog_manager.cc:1347] Generated new cluster ID: fdee9a8b2e86483c97d1f2785168633f
I20260811 16:08:29.324083 9440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:29.362819 9440 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:29.364544 9440 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:29.376873 9440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b: Generated new TSK 0
I20260811 16:08:29.377581 9440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:29.382285 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:29.388393 9461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.389515 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.391062 9467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.393445 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:29.394279 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:29.394461 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:29.394575 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464509394566 us; error 0 us; skew 500 ppm
I20260811 16:08:29.395022 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:29.397459 8080 webserver.cc:460] Webserver started at http://127.7.228.1:39479/ using document root <none> and password file <none>
I20260811 16:08:29.397956 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:29.398133 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:29.398332 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:29.399549 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "6d1a72c68994451b83a9b1c2d585e09f"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.404538 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 16:08:29.407995 9475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.408985 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:08:29.409298 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "6d1a72c68994451b83a9b1c2d585e09f"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.409633 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:29.425568 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:29.426738 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:29.428169 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:29.430612 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:29.430804 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.431061 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:29.431227 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.475189 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:43365
I20260811 16:08:29.475275 9589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:43365 every 8 connection(s)
I20260811 16:08:29.480250 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:29.489756 9597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.489904 9596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.490690 9601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.491125 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:29.492152 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:29.492422 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:29.492702 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464509492608 us; error 0 us; skew 500 ppm
I20260811 16:08:29.493399 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:29.496595 8080 webserver.cc:460] Webserver started at http://127.7.228.2:32919/ using document root <none> and password file <none>
I20260811 16:08:29.497285 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:29.497503 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:29.497861 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:29.497888 9590 heartbeater.cc:347] Connected to a master server at 127.7.228.62:35801
I20260811 16:08:29.498344 9590 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:29.499114 9590 heartbeater.cc:511] Master 127.7.228.62:35801 requested a full tablet report, sending...
I20260811 16:08:29.499220 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "98dbbc11553d4a8a9c15ea2653dc27f9"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.501494 9374 ts_manager.cc:194] Registered new tserver with Master: 6d1a72c68994451b83a9b1c2d585e09f (127.7.228.1:43365)
I20260811 16:08:29.504164 9374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39704
I20260811 16:08:29.505462 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 16:08:29.509644 9610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.510490 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 16:08:29.510795 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "98dbbc11553d4a8a9c15ea2653dc27f9"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.511176 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:29.530431 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:29.531652 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:29.533074 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:29.535315 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:29.535508 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.535748 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:29.535888 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.576377 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:33745
I20260811 16:08:29.576467 9706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:33745 every 8 connection(s)
I20260811 16:08:29.580924 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:29.588819 9713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:29.589957 9714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.595494 9707 heartbeater.cc:347] Connected to a master server at 127.7.228.62:35801
I20260811 16:08:29.595685 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:29.595587 9717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:29.596009 9707 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:29.596815 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260811 16:08:29.596817 9707 heartbeater.cc:511] Master 127.7.228.62:35801 requested a full tablet report, sending...
W20260811 16:08:29.597146 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:29.597344 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464509597328 us; error 0 us; skew 500 ppm
I20260811 16:08:29.597973 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:29.599095 9374 ts_manager.cc:194] Registered new tserver with Master: 98dbbc11553d4a8a9c15ea2653dc27f9 (127.7.228.2:33745)
I20260811 16:08:29.600739 8080 webserver.cc:460] Webserver started at http://127.7.228.3:43757/ using document root <none> and password file <none>
I20260811 16:08:29.601033 9374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39712
I20260811 16:08:29.601367 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:29.601588 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:29.601845 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:29.603013 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-2-root/instance:
uuid: "6bc000ab53654866bee27b34ebfb2c83"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.607692 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 16:08:29.611179 9727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.612018 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:08:29.612313 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-2-root
uuid: "6bc000ab53654866bee27b34ebfb2c83"
format_stamp: "Formatted at 2026-08-11 16:08:29 on dist-test-slave-ncp9"
I20260811 16:08:29.612583 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786464498564520-8080-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:29.624954 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:29.626190 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:29.627683 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:29.630509 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:29.630753 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.630998 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:29.631146 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:29.701500 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.3:40795
I20260811 16:08:29.701617 9817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.3:40795 every 8 connection(s)
I20260811 16:08:29.720386 9818 heartbeater.cc:347] Connected to a master server at 127.7.228.62:35801
I20260811 16:08:29.720748 9818 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:29.721475 9818 heartbeater.cc:511] Master 127.7.228.62:35801 requested a full tablet report, sending...
I20260811 16:08:29.723555 9374 ts_manager.cc:194] Registered new tserver with Master: 6bc000ab53654866bee27b34ebfb2c83 (127.7.228.3:40795)
I20260811 16:08:29.724110 8080 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017776522s
I20260811 16:08:29.725776 9374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39728
I20260811 16:08:29.753330 9374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260811 16:08:29.754889 9374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 16:08:29.803059 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 74b1def29f4f4b3bb23fc7cb812489b5 (DEFAULT_TABLE table=test-table [id=ea166d5bd2a84afdb9e9a4fe3e5fe46d]), partition=
I20260811 16:08:29.804304 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b1def29f4f4b3bb23fc7cb812489b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:29.809854 9773 tablet_service.cc:1467] Processing CreateTablet for tablet 74b1def29f4f4b3bb23fc7cb812489b5 (DEFAULT_TABLE table=test-table [id=ea166d5bd2a84afdb9e9a4fe3e5fe46d]), partition=
I20260811 16:08:29.811031 9773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b1def29f4f4b3bb23fc7cb812489b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:29.814033 9654 tablet_service.cc:1467] Processing CreateTablet for tablet 74b1def29f4f4b3bb23fc7cb812489b5 (DEFAULT_TABLE table=test-table [id=ea166d5bd2a84afdb9e9a4fe3e5fe46d]), partition=
I20260811 16:08:29.815238 9654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b1def29f4f4b3bb23fc7cb812489b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:29.821233 9850 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: Bootstrap starting.
I20260811 16:08:29.826289 9850 tablet_bootstrap.cc:654] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:29.829902 9851 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: Bootstrap starting.
I20260811 16:08:29.833566 9850 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: No bootstrap required, opened a new log
I20260811 16:08:29.833986 9850 ts_tablet_manager.cc:1397] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260811 16:08:29.836400 9850 raft_consensus.cc:348] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.837055 9851 tablet_bootstrap.cc:654] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:29.837972 9854 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: Bootstrap starting.
I20260811 16:08:29.838088 9850 raft_consensus.cc:374] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:29.838465 9850 raft_consensus.cc:729] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d1a72c68994451b83a9b1c2d585e09f, State: Initialized, Role: FOLLOWER
I20260811 16:08:29.839221 9850 consensus_queue.cc:260] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.843190 9590 heartbeater.cc:503] Master 127.7.228.62:35801 was elected leader, sending a full tablet report...
I20260811 16:08:29.843690 9850 ts_tablet_manager.cc:1428] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260811 16:08:29.844022 9854 tablet_bootstrap.cc:654] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:29.851399 9851 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: No bootstrap required, opened a new log
I20260811 16:08:29.851847 9851 ts_tablet_manager.cc:1397] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.012s
I20260811 16:08:29.853724 9854 tablet_bootstrap.cc:492] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: No bootstrap required, opened a new log
I20260811 16:08:29.854192 9854 ts_tablet_manager.cc:1397] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260811 16:08:29.854063 9851 raft_consensus.cc:348] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.854743 9851 raft_consensus.cc:374] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:29.854988 9851 raft_consensus.cc:729] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc000ab53654866bee27b34ebfb2c83, State: Initialized, Role: FOLLOWER
I20260811 16:08:29.855746 9851 consensus_queue.cc:260] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.856791 9854 raft_consensus.cc:348] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.857481 9854 raft_consensus.cc:374] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:29.857717 9854 raft_consensus.cc:729] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dbbc11553d4a8a9c15ea2653dc27f9, State: Initialized, Role: FOLLOWER
I20260811 16:08:29.858399 9854 consensus_queue.cc:260] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.858978 9818 heartbeater.cc:503] Master 127.7.228.62:35801 was elected leader, sending a full tablet report...
I20260811 16:08:29.859274 9851 ts_tablet_manager.cc:1428] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 16:08:29.861641 9707 heartbeater.cc:503] Master 127.7.228.62:35801 was elected leader, sending a full tablet report...
I20260811 16:08:29.862066 9854 ts_tablet_manager.cc:1428] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 16:08:29.904042 9860 raft_consensus.cc:482] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:29.904471 9860 raft_consensus.cc:504] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.906571 9860 leader_election.cc:288] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d1a72c68994451b83a9b1c2d585e09f (127.7.228.1:43365), 98dbbc11553d4a8a9c15ea2653dc27f9 (127.7.228.2:33745)
I20260811 16:08:29.919628 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6bc000ab53654866bee27b34ebfb2c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d1a72c68994451b83a9b1c2d585e09f" is_pre_election: true
I20260811 16:08:29.920548 9855 raft_consensus.cc:482] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:29.921130 9855 raft_consensus.cc:504] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.922385 9547 raft_consensus.cc:2454] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bc000ab53654866bee27b34ebfb2c83 in term 0.
I20260811 16:08:29.922557 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6bc000ab53654866bee27b34ebfb2c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" is_pre_election: true
I20260811 16:08:29.923370 9677 raft_consensus.cc:2454] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bc000ab53654866bee27b34ebfb2c83 in term 0.
I20260811 16:08:29.923879 9855 leader_election.cc:288] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bc000ab53654866bee27b34ebfb2c83 (127.7.228.3:40795), 98dbbc11553d4a8a9c15ea2653dc27f9 (127.7.228.2:33745)
I20260811 16:08:29.925382 9734 leader_election.cc:302] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bc000ab53654866bee27b34ebfb2c83, 98dbbc11553d4a8a9c15ea2653dc27f9; no voters:
I20260811 16:08:29.926452 9860 raft_consensus.cc:2781] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:29.926824 9860 raft_consensus.cc:482] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:29.927146 9860 raft_consensus.cc:3037] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:29.933799 9860 raft_consensus.cc:504] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.935843 9860 leader_election.cc:288] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [CANDIDATE]: Term 1 election: Requested vote from peers 6d1a72c68994451b83a9b1c2d585e09f (127.7.228.1:43365), 98dbbc11553d4a8a9c15ea2653dc27f9 (127.7.228.2:33745)
I20260811 16:08:29.936695 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6bc000ab53654866bee27b34ebfb2c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d1a72c68994451b83a9b1c2d585e09f"
I20260811 16:08:29.937131 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6bc000ab53654866bee27b34ebfb2c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9"
I20260811 16:08:29.937369 9547 raft_consensus.cc:3037] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:29.937704 9677 raft_consensus.cc:3037] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:29.943216 9676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6d1a72c68994451b83a9b1c2d585e09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" is_pre_election: true
I20260811 16:08:29.944521 9677 raft_consensus.cc:2454] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bc000ab53654866bee27b34ebfb2c83 in term 1.
I20260811 16:08:29.944526 9547 raft_consensus.cc:2454] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bc000ab53654866bee27b34ebfb2c83 in term 1.
I20260811 16:08:29.945904 9734 leader_election.cc:302] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bc000ab53654866bee27b34ebfb2c83, 98dbbc11553d4a8a9c15ea2653dc27f9; no voters:
I20260811 16:08:29.947083 9860 raft_consensus.cc:2781] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:29.947774 9784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b1def29f4f4b3bb23fc7cb812489b5" candidate_uuid: "6d1a72c68994451b83a9b1c2d585e09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bc000ab53654866bee27b34ebfb2c83" is_pre_election: true
I20260811 16:08:29.949076 9860 raft_consensus.cc:686] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 1 LEADER]: Becoming Leader. State: Replica: 6bc000ab53654866bee27b34ebfb2c83, State: Running, Role: LEADER
I20260811 16:08:29.949764 9860 consensus_queue.cc:237] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } }
I20260811 16:08:29.955380 9481 leader_election.cc:302] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d1a72c68994451b83a9b1c2d585e09f; no voters: 6bc000ab53654866bee27b34ebfb2c83, 98dbbc11553d4a8a9c15ea2653dc27f9
I20260811 16:08:29.956295 9855 raft_consensus.cc:2726] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 16:08:29.957904 9372 catalog_manager.cc:5495] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bc000ab53654866bee27b34ebfb2c83 (127.7.228.3). New cstate: current_term: 1 leader_uuid: "6bc000ab53654866bee27b34ebfb2c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bc000ab53654866bee27b34ebfb2c83" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 40795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:30.457734 9871 consensus_queue.cc:1035] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d1a72c68994451b83a9b1c2d585e09f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 43365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:30.472229 9878 consensus_queue.cc:1035] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98dbbc11553d4a8a9c15ea2653dc27f9" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 33745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:31.056928 8080 tablet_replica.cc:330] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f: stopping tablet replica
I20260811 16:08:31.057658 8080 raft_consensus.cc:2229] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:08:31.058166 8080 raft_consensus.cc:2258] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6d1a72c68994451b83a9b1c2d585e09f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 16:08:31.058476 9728 consensus_peers.cc:487] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 -> Peer 6d1a72c68994451b83a9b1c2d585e09f (127.7.228.1:43365): Couldn't send request to peer 6d1a72c68994451b83a9b1c2d585e09f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:08:31.614743 9728 consensus_queue.cc:579] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [LEADER]: The tablet replica hosted on peer 6d1a72c68994451b83a9b1c2d585e09f has failed
I20260811 16:08:32.096467 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:08:32.113140 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:32.130894 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:08:32.139381 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:08:32.155555 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:32.156193 8080 tablet_replica.cc:330] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9: stopping tablet replica
I20260811 16:08:32.156710 8080 raft_consensus.cc:2229] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:08:32.157142 8080 raft_consensus.cc:2258] T 74b1def29f4f4b3bb23fc7cb812489b5 P 98dbbc11553d4a8a9c15ea2653dc27f9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 16:08:32.157902 9728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.228.1:43365: connect: Connection refused (error 111)
I20260811 16:08:32.175402 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:08:32.183602 8080 tablet_server.cc:178] TabletServer@127.7.228.3:0 shutting down...
I20260811 16:08:32.199563 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:32.200129 8080 tablet_replica.cc:330] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83: stopping tablet replica
I20260811 16:08:32.200641 8080 raft_consensus.cc:2229] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:32.201434 8080 raft_consensus.cc:2258] T 74b1def29f4f4b3bb23fc7cb812489b5 P 6bc000ab53654866bee27b34ebfb2c83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:32.219774 8080 tablet_server.cc:194] TabletServer@127.7.228.3:0 shutdown complete.
I20260811 16:08:32.228520 8080 master.cc:537] Master@127.7.228.62:35801 shutting down...
I20260811 16:08:32.243201 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:32.243678 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:32.243916 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P cd56acffb9024d27b1b2b777fc49960b: stopping tablet replica
I20260811 16:08:32.261610 8080 master.cc:552] Master@127.7.228.62:35801 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (3116 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260811 16:08:32.286759 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:43339
I20260811 16:08:32.288072 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:32.293567 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.294909 9904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:32.300415 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:32.301538 9906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:32.302421 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:32.302652 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:32.302801 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464512302784 us; error 0 us; skew 500 ppm
I20260811 16:08:32.303323 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:32.310266 8080 webserver.cc:460] Webserver started at http://127.7.228.62:33311/ using document root <none> and password file <none>
I20260811 16:08:32.310802 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:32.310989 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:32.311237 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:32.312361 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "f291913d485a4d91aea1c18b157d5f8d"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.317076 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:08:32.320668 9912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.321571 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:08:32.321884 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "f291913d485a4d91aea1c18b157d5f8d"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.322194 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:32.345980 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:32.347019 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:32.384012 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:43339
I20260811 16:08:32.384142 9992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:43339 every 8 connection(s)
I20260811 16:08:32.387784 9993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:32.397774 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d: Bootstrap starting.
I20260811 16:08:32.401765 9993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:32.405834 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d: No bootstrap required, opened a new log
I20260811 16:08:32.407719 9993 raft_consensus.cc:348] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER }
I20260811 16:08:32.408109 9993 raft_consensus.cc:374] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:32.408344 9993 raft_consensus.cc:729] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f291913d485a4d91aea1c18b157d5f8d, State: Initialized, Role: FOLLOWER
I20260811 16:08:32.408921 9993 consensus_queue.cc:260] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER }
I20260811 16:08:32.409354 9993 raft_consensus.cc:388] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:08:32.409574 9993 raft_consensus.cc:482] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:08:32.409822 9993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:32.414099 9993 raft_consensus.cc:504] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER }
I20260811 16:08:32.414633 9993 leader_election.cc:302] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f291913d485a4d91aea1c18b157d5f8d; no voters:
I20260811 16:08:32.415802 9993 leader_election.cc:288] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:08:32.416113 9996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:32.417434 9996 raft_consensus.cc:686] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 1 LEADER]: Becoming Leader. State: Replica: f291913d485a4d91aea1c18b157d5f8d, State: Running, Role: LEADER
I20260811 16:08:32.418134 9996 consensus_queue.cc:237] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER }
I20260811 16:08:32.418711 9993 sys_catalog.cc:564] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:32.421295 10000 sys_catalog.cc:455] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f291913d485a4d91aea1c18b157d5f8d. Latest consensus state: current_term: 1 leader_uuid: "f291913d485a4d91aea1c18b157d5f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER } }
I20260811 16:08:32.421288 9999 sys_catalog.cc:455] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f291913d485a4d91aea1c18b157d5f8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f291913d485a4d91aea1c18b157d5f8d" member_type: VOTER } }
I20260811 16:08:32.422092 9999 sys_catalog.cc:458] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:32.422013 10000 sys_catalog.cc:458] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:32.429348 10006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:32.436443 10006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:32.437609 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:32.445470 10006 catalog_manager.cc:1347] Generated new cluster ID: 446d3e2f3bda4787b6015939d8c3b1a0
I20260811 16:08:32.445813 10006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:32.480628 10006 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:32.482406 10006 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:32.495385 10006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d: Generated new TSK 0
I20260811 16:08:32.496084 10006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:32.504663 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:32.511055 10022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.512042 10024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.513590 10027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:32.514009 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:32.514868 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:32.515081 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:32.515239 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464512515223 us; error 0 us; skew 500 ppm
I20260811 16:08:32.515744 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:32.518299 8080 webserver.cc:460] Webserver started at http://127.7.228.1:37075/ using document root <none> and password file <none>
I20260811 16:08:32.518821 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:32.519032 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:32.519306 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:32.520562 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "59c9b71ea44540fcbc35472a9d8f2673"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.526038 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 16:08:32.530176 10035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.531121 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 16:08:32.531464 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "59c9b71ea44540fcbc35472a9d8f2673"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.531773 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:32.547277 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:32.548475 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:32.550137 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:32.552583 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:32.552804 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.553086 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:32.553238 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.594981 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:37429
I20260811 16:08:32.595084 10146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:37429 every 8 connection(s)
I20260811 16:08:32.611811 10149 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43339
I20260811 16:08:32.612188 10149 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:32.612849 10149 heartbeater.cc:511] Master 127.7.228.62:43339 requested a full tablet report, sending...
I20260811 16:08:32.614840 9940 ts_manager.cc:194] Registered new tserver with Master: 59c9b71ea44540fcbc35472a9d8f2673 (127.7.228.1:37429)
I20260811 16:08:32.615023 8080 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016696307s
I20260811 16:08:32.616834 9940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59870
I20260811 16:08:32.639544 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:08:32.657708 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:32.673650 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:08:32.681748 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 16:08:32.684603 8080 tablet_server.cc:93] deduplicated master addresses: 127.7.228.62:43339
W20260811 16:08:32.688125 10164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.688655 10167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.689957 10169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:32.691166 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:32.692226 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:32.692446 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:32.692598 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464512692586 us; error 0 us; skew 500 ppm
I20260811 16:08:32.693222 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:32.695953 8080 webserver.cc:460] Webserver started at http://127.7.228.1:37075/ using document root <none> and password file <none>
I20260811 16:08:32.696455 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:32.696621 8080 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260811 16:08:32.700387 8080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:08:32.703433 10176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.704281 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:08:32.704592 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "59c9b71ea44540fcbc35472a9d8f2673"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.704933 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:32.717887 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:32.718951 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:32.720579 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:32.723080 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:32.723341 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.723546 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:32.723678 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.768275 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:37429
I20260811 16:08:32.768378 10274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:37429 every 8 connection(s)
I20260811 16:08:32.774314 8080 tablet_server.cc:178] TabletServer@127.7.228.1:37429 shutting down...
I20260811 16:08:32.804668 10275 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43339
I20260811 16:08:32.805120 10275 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:32.805838 10275 heartbeater.cc:511] Master 127.7.228.62:43339 requested a full tablet report, sending...
I20260811 16:08:32.807991 9940 ts_manager.cc:194] Re-registered known tserver with Master: 59c9b71ea44540fcbc35472a9d8f2673 (127.7.228.1:37429)
I20260811 16:08:32.809566 9940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59878
I20260811 16:08:32.812485 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:08:32.829474 8080 tablet_server.cc:194] TabletServer@127.7.228.1:37429 shutdown complete.
I20260811 16:08:32.837033 8080 master.cc:537] Master@127.7.228.62:43339 shutting down...
I20260811 16:08:32.851432 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:08:32.851991 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:32.852393 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P f291913d485a4d91aea1c18b157d5f8d: stopping tablet replica
I20260811 16:08:32.872159 8080 master.cc:552] Master@127.7.228.62:43339 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (608 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260811 16:08:32.895082 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:43139,127.7.228.61:34181,127.7.228.60:38613
I20260811 16:08:32.896385 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:32.901434 10291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.902513 10292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:32.904752 10295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:32.905153 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:32.905898 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:32.906077 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:32.906222 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464512906212 us; error 0 us; skew 500 ppm
I20260811 16:08:32.906765 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:32.909222 8080 webserver.cc:460] Webserver started at http://127.7.228.62:42581/ using document root <none> and password file <none>
I20260811 16:08:32.909709 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:32.909878 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:32.910126 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:32.911209 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "a17ce76946324f1d9576e7006ab317cc"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.915916 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 16:08:32.919478 10303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:32.920262 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 16:08:32.920540 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "a17ce76946324f1d9576e7006ab317cc"
format_stamp: "Formatted at 2026-08-11 16:08:32 on dist-test-slave-ncp9"
I20260811 16:08:32.920848 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:32.942409 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:32.943471 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:32.980738 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:43139
I20260811 16:08:32.980851 10384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:43139 every 8 connection(s)
I20260811 16:08:32.986131 10385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:32.992919 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 16:08:32.995616 10385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
W20260811 16:08:32.999042 10391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.003300 10392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.004804 10398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.006218 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:33.007200 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:33.007457 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:33.007630 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464513007616 us; error 0 us; skew 500 ppm
I20260811 16:08:33.008304 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:33.011053 8080 webserver.cc:460] Webserver started at http://127.7.228.61:35997/ using document root <none> and password file <none>
I20260811 16:08:33.011525 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:33.011705 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:33.011950 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:33.013041 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-1-root/instance:
uuid: "9f927cbce0704eac8a7d3836460dce87"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.014647 10385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.017705 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
W20260811 16:08:33.019341 10385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.61:34181: Network error: Client connection negotiation failed: client connection to 127.7.228.61:34181: connect: Connection refused (error 111)
I20260811 16:08:33.021507 10407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.022351 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260811 16:08:33.022642 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-1-root
uuid: "9f927cbce0704eac8a7d3836460dce87"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.022938 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:33.040856 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:33.041906 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:33.065002 10385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } attempt: 1
W20260811 16:08:33.069607 10385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.61:34181: Network error: Client connection negotiation failed: client connection to 127.7.228.61:34181: connect: Connection refused (error 111)
I20260811 16:08:33.080796 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.61:34181
I20260811 16:08:33.080886 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.61:34181 every 8 connection(s)
I20260811 16:08:33.084626 10484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:33.084692 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:33.090114 10486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.091629 10487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.093104 10484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.095378 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:33.096681 10492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.097523 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:33.097743 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:33.097900 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464513097888 us; error 0 us; skew 500 ppm
I20260811 16:08:33.098429 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:33.101413 8080 webserver.cc:460] Webserver started at http://127.7.228.60:43717/ using document root <none> and password file <none>
I20260811 16:08:33.102066 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:33.102344 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:33.102650 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:33.103986 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root/instance:
uuid: "7050dea6bf96493a9a5ebfa411565d7f"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.104061 10385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } attempt: 2
I20260811 16:08:33.108850 10484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.110119 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 16:08:33.115568 10508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.116573 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 16:08:33.116874 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
uuid: "7050dea6bf96493a9a5ebfa411565d7f"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.117280 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:33.126127 10385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } has no permanent_uuid. Determining permanent_uuid...
W20260811 16:08:33.130870 10385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.60:38613: Network error: Client connection negotiation failed: client connection to 127.7.228.60:38613: connect: Connection refused (error 111)
I20260811 16:08:33.133723 10484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } has no permanent_uuid. Determining permanent_uuid...
W20260811 16:08:33.139297 10484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.60:38613: Network error: Client connection negotiation failed: client connection to 127.7.228.60:38613: connect: Connection refused (error 111)
I20260811 16:08:33.141518 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:33.142583 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:33.163815 10484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } attempt: 1
W20260811 16:08:33.167805 10484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.60:38613: Network error: Client connection negotiation failed: client connection to 127.7.228.60:38613: connect: Connection refused (error 111)
I20260811 16:08:33.169301 10385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } attempt: 1
W20260811 16:08:33.173799 10385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.228.60:38613: Network error: Client connection negotiation failed: client connection to 127.7.228.60:38613: connect: Connection refused (error 111)
I20260811 16:08:33.182093 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.60:38613
I20260811 16:08:33.182211 10584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.60:38613 every 8 connection(s)
I20260811 16:08:33.185330 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:08:33.186125 10585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:33.192004 10585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.204917 10585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.216288 10585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } has no permanent_uuid. Determining permanent_uuid...
I20260811 16:08:33.219290 10484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } attempt: 2
I20260811 16:08:33.223343 10385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } attempt: 2
I20260811 16:08:33.244284 10484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: Bootstrap starting.
I20260811 16:08:33.245535 10585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Bootstrap starting.
I20260811 16:08:33.248616 10484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:33.249604 10385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: Bootstrap starting.
I20260811 16:08:33.251865 10585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:33.253146 10484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: No bootstrap required, opened a new log
I20260811 16:08:33.255434 10484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.255963 10385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:33.256214 10484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:33.256508 10484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f927cbce0704eac8a7d3836460dce87, State: Initialized, Role: FOLLOWER
I20260811 16:08:33.257181 10484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.257682 10585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: No bootstrap required, opened a new log
I20260811 16:08:33.259161 10596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.260172 10596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.260800 10484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:33.260772 10585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.261647 10585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:33.261929 10585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Initialized, Role: FOLLOWER
I20260811 16:08:33.262702 10585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.263521 10385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: No bootstrap required, opened a new log
I20260811 16:08:33.265187 10602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.266242 10602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.266855 10585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:33.266512 10385 raft_consensus.cc:348] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.267309 10385 raft_consensus.cc:374] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:33.267627 10385 raft_consensus.cc:729] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17ce76946324f1d9576e7006ab317cc, State: Initialized, Role: FOLLOWER
I20260811 16:08:33.268397 10385 consensus_queue.cc:260] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.270788 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.271858 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.272832 10385 sys_catalog.cc:564] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:33.278128 10602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:33.278755 10602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.281318 10602 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:33.282883 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:33.283922 10450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 0.
I20260811 16:08:33.282881 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:33.285063 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 0.
I20260811 16:08:33.286373 10511 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:33.287390 10602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:33.287804 10602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:33.288110 10602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:33.296007 10602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.299722 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc"
I20260811 16:08:33.300439 10342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:33.300536 10602 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 1 election: Requested vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:33.303305 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87"
I20260811 16:08:33.303786 10450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:33.308414 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 1.
I20260811 16:08:33.311019 10511 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:33.311754 10450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 1.
I20260811 16:08:33.312163 10602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 FOLLOWER]: Leader election won for term 1
W20260811 16:08:33.320732 10650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 16:08:33.321168 10650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 16:08:33.322610 10602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 LEADER]: Becoming Leader. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Running, Role: LEADER
I20260811 16:08:33.323716 10602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:33.334674 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260811 16:08:33.335031 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260811 16:08:33.334772 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.335525 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:33.338384 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:33.339793 10662 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 16:08:33.340081 10662 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 16:08:33.340348 10659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 16:08:33.344074 10658 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 16:08:33.344480 10658 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 16:08:33.346027 10659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 16:08:33.347107 10663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.348832 10664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.350118 10667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.351536 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:33.352576 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:33.352886 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:33.353132 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464513353106 us; error 0 us; skew 500 ppm
I20260811 16:08:33.353683 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:33.357393 8080 webserver.cc:460] Webserver started at http://127.7.228.1:42543/ using document root <none> and password file <none>
I20260811 16:08:33.357443 10342 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Refusing update from remote peer 7050dea6bf96493a9a5ebfa411565d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 16:08:33.357774 10450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Refusing update from remote peer 7050dea6bf96493a9a5ebfa411565d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 16:08:33.358219 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:33.358492 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:33.358844 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:33.358935 10602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:33.359745 10647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:33.360061 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.374523 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.375366 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.377951 10596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.378764 10596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.382095 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.382334 10602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.382933 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:33.383050 10602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:33.383605 10686 mvcc.cc:205] Tried to move back new op lower bound from 7317358646699204608 to 7317358646584877056. Current Snapshot: MvccSnapshot[applied={T|T < 7317358646699204608}]
I20260811 16:08:33.385672 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.013s sys 0.008s
I20260811 16:08:33.385233 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.386035 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.390231 10596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:33.391114 10596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:33.391374 10694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.391654 10659 catalog_manager.cc:1347] Generated new cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:33.392089 10659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:33.392531 8080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260811 16:08:33.392845 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.393199 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:33.407487 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:33.408653 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:33.410444 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:33.412937 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:33.413163 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.413386 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:33.413530 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.429637 10659 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:08:33.432104 10659 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:33.458901 10659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Generated new TSK 0
I20260811 16:08:33.459734 10659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:33.486800 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:34591
I20260811 16:08:33.486887 10801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:34591 every 8 connection(s)
I20260811 16:08:33.500005 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 16:08:33.508822 10802 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:33.509372 10802 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.510373 10802 heartbeater.cc:511] Master 127.7.228.60:38613 requested a full tablet report, sending...
I20260811 16:08:33.512207 10803 heartbeater.cc:347] Connected to a master server at 127.7.228.61:34181
I20260811 16:08:33.512610 10803 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.512984 10533 ts_manager.cc:194] Registered new tserver with Master: d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:33.513572 10803 heartbeater.cc:511] Master 127.7.228.61:34181 requested a full tablet report, sending...
I20260811 16:08:33.516207 10533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41918
I20260811 16:08:33.517040 10432 ts_manager.cc:194] Registered new tserver with Master: d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:33.521770 10804 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43139
I20260811 16:08:33.522203 10804 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.522994 10804 heartbeater.cc:511] Master 127.7.228.62:43139 requested a full tablet report, sending...
W20260811 16:08:33.523375 10815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.523957 8080 server_base.cc:1034] running on GCE node
W20260811 16:08:33.522238 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.525022 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.525988 10329 ts_manager.cc:194] Registered new tserver with Master: d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:33.526136 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:33.526397 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:33.526547 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464513526534 us; error 0 us; skew 500 ppm
I20260811 16:08:33.527259 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:33.529831 8080 webserver.cc:460] Webserver started at http://127.7.228.2:44651/ using document root <none> and password file <none>
I20260811 16:08:33.530326 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:33.530517 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:33.530766 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:33.531824 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "52ad56aab4d843c599224c82b3f41311"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.536269 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:08:33.539952 10828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.540755 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:08:33.541100 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "52ad56aab4d843c599224c82b3f41311"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.541399 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:33.556005 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:33.557272 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:33.559319 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:33.562366 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:33.562597 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.562827 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:33.563019 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.622984 10802 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:33.643323 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:40359
I20260811 16:08:33.643440 10923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:40359 every 8 connection(s)
I20260811 16:08:33.655480 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:33.667976 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:33.671365 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.677047 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:33.677660 10929 heartbeater.cc:347] Connected to a master server at 127.7.228.61:34181
I20260811 16:08:33.678817 10929 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.679766 10929 heartbeater.cc:511] Master 127.7.228.61:34181 requested a full tablet report, sending...
I20260811 16:08:33.679955 10925 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:33.680457 10925 heartbeater.cc:464] Registering TS with master...
W20260811 16:08:33.680593 10943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:33.681672 10925 heartbeater.cc:511] Master 127.7.228.60:38613 requested a full tablet report, sending...
I20260811 16:08:33.682260 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:33.682590 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:33.682844 10432 ts_manager.cc:194] Registered new tserver with Master: 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:33.682839 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464513682821 us; error 0 us; skew 500 ppm
I20260811 16:08:33.684738 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:33.686538 10533 ts_manager.cc:194] Registered new tserver with Master: 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:33.689138 8080 webserver.cc:460] Webserver started at http://127.7.228.3:36159/ using document root <none> and password file <none>
I20260811 16:08:33.689299 10930 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43139
I20260811 16:08:33.689724 10930 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.689903 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:33.690197 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:08:33.690538 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:08:33.690666 10930 heartbeater.cc:511] Master 127.7.228.62:43139 requested a full tablet report, sending...
I20260811 16:08:33.690698 10533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41932
I20260811 16:08:33.692099 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-2-root/instance:
uuid: "429744fd0c424b86bef5550dccfc6530"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.693622 10329 ts_manager.cc:194] Registered new tserver with Master: 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:33.697928 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 16:08:33.702766 10952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.703912 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 16:08:33.704267 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-2-root
uuid: "429744fd0c424b86bef5550dccfc6530"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:33.704571 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:33.724968 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:33.726441 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:08:33.728757 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:08:33.732138 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:08:33.732409 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.732707 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:08:33.732869 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:33.794819 10925 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:33.810897 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.3:37155
I20260811 16:08:33.810997 11058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.3:37155 every 8 connection(s)
I20260811 16:08:33.849571 11066 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:33.850121 11066 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.851269 11066 heartbeater.cc:511] Master 127.7.228.60:38613 requested a full tablet report, sending...
I20260811 16:08:33.856595 10533 ts_manager.cc:194] Registered new tserver with Master: 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:33.863097 10533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41940
I20260811 16:08:33.875305 11070 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43139
I20260811 16:08:33.875773 11070 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.876859 11070 heartbeater.cc:511] Master 127.7.228.62:43139 requested a full tablet report, sending...
I20260811 16:08:33.880203 11067 heartbeater.cc:347] Connected to a master server at 127.7.228.61:34181
I20260811 16:08:33.880620 11067 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:33.880755 10329 ts_manager.cc:194] Registered new tserver with Master: 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:33.881685 11067 heartbeater.cc:511] Master 127.7.228.61:34181 requested a full tablet report, sending...
I20260811 16:08:33.885426 10432 ts_manager.cc:194] Registered new tserver with Master: 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:33.893541 8080 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.000648991s
I20260811 16:08:33.972985 11066 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:34.073963 11088 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:34.075398 11086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:34.077370 11086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:34.076102 11088 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:34.090813 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:34.091609 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f927cbce0704eac8a7d3836460dce87 in term 1.
I20260811 16:08:34.093921 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:34.095825 11086 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:34.097386 11086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 16:08:34.098600 11086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:34.099660 11086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:34.096519 11088 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:34.115914 11086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:34.120671 11086 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 2 election: Requested vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:34.121797 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc"
I20260811 16:08:34.122493 10342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:34.130336 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f927cbce0704eac8a7d3836460dce87 in term 2.
I20260811 16:08:34.131932 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:34.132813 11086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:08:34.133342 11086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 LEADER]: Becoming Leader. State: Replica: 9f927cbce0704eac8a7d3836460dce87, State: Running, Role: LEADER
I20260811 16:08:34.134204 11086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:34.137905 11089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.138815 11089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:34.158581 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:34.177670 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:34.179268 10450 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 LEADER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 1. Current term is 2. Ops: []
I20260811 16:08:34.179523 10305 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:34.180368 11088 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:34.271656 10647 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 16:08:34.274338 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:34.276002 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f"
I20260811 16:08:34.277467 10342 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 1. Current term is 2. Ops: []
I20260811 16:08:34.278928 10341 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Refusing update from remote peer 9f927cbce0704eac8a7d3836460dce87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260811 16:08:34.280809 11089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 16:08:34.289477 11088 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.290290 11088 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:34.293395 10647 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer 9f927cbce0704eac8a7d3836460dce87 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260811 16:08:34.294152 10647 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 16:08:34.310613 10550 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:34.311538 10550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Running, Role: LEADER
I20260811 16:08:34.312518 10550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:34.321287 11102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:34.325358 11089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.326303 11089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
W20260811 16:08:34.331022 10511 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.62:43139, user_credentials={real_user=slave}} blocked reactor thread for 48138.7us
I20260811 16:08:34.332926 10550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:34.336726 11102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:34.336791 11088 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.337702 11088 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:34.340965 11102 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:34.341262 11102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:34.343137 10550 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Refusing update from remote peer 9f927cbce0704eac8a7d3836460dce87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260811 16:08:34.344020 11104 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260811 16:08:34.348063 11102 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:34.348776 10658 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:34.349104 10658 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: loading cluster ID for follower catalog manager: success
I20260811 16:08:34.349913 11086 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 16:08:34.352113 10662 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 16:08:34.352572 11102 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: Loaded TSK: 0
I20260811 16:08:34.354070 11102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:34.355777 10658 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: acquiring CA information for follower catalog manager: success
I20260811 16:08:34.360239 10658 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 16:08:34.364326 11105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.364537 11104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.365269 11105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:34.365473 11104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:34.366783 10432 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260811 16:08:34.368528 10432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 16:08:34.380548 11086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:34.381465 11086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:34.460366 10739 tablet_service.cc:1467] Processing CreateTablet for tablet 84e71aac7b9b46479a8ce87b88b74e45 (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.462158 10739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e71aac7b9b46479a8ce87b88b74e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.462445 10734 tablet_service.cc:1467] Processing CreateTablet for tablet 6afe0fe649d34850a77793d9372f92fb (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.463830 10734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afe0fe649d34850a77793d9372f92fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.467909 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 84e71aac7b9b46479a8ce87b88b74e45 (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.461041 10738 tablet_service.cc:1467] Processing CreateTablet for tablet d42b5e0273654c95badc4250fc283cbd (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.469295 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e71aac7b9b46479a8ce87b88b74e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.469710 10738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42b5e0273654c95badc4250fc283cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.470065 10998 tablet_service.cc:1467] Processing CreateTablet for tablet d42b5e0273654c95badc4250fc283cbd (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.471499 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42b5e0273654c95badc4250fc283cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.472196 10864 tablet_service.cc:1467] Processing CreateTablet for tablet 6afe0fe649d34850a77793d9372f92fb (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.473664 10864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afe0fe649d34850a77793d9372f92fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.474145 10997 tablet_service.cc:1467] Processing CreateTablet for tablet 6afe0fe649d34850a77793d9372f92fb (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.475606 10997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6afe0fe649d34850a77793d9372f92fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.474074 10865 tablet_service.cc:1467] Processing CreateTablet for tablet d42b5e0273654c95badc4250fc283cbd (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.481658 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42b5e0273654c95badc4250fc283cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.468549 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 84e71aac7b9b46479a8ce87b88b74e45 (DEFAULT_TABLE table=test-table [id=465c3f203d8b47ba84eb9663bfecfe23]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 16:08:34.486135 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e71aac7b9b46479a8ce87b88b74e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:08:34.491186 10803 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:34.501649 10929 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:34.512293 11067 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:34.551476 11130 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: Bootstrap starting.
I20260811 16:08:34.552584 11127 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: Bootstrap starting.
I20260811 16:08:34.552810 11126 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Bootstrap starting.
I20260811 16:08:34.560056 11130 tablet_bootstrap.cc:654] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.560380 11126 tablet_bootstrap.cc:654] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.560786 11127 tablet_bootstrap.cc:654] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.567852 11130 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: No bootstrap required, opened a new log
I20260811 16:08:34.568418 11130 ts_tablet_manager.cc:1397] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.003s
I20260811 16:08:34.568702 11127 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: No bootstrap required, opened a new log
I20260811 16:08:34.569175 11127 ts_tablet_manager.cc:1397] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.012s
I20260811 16:08:34.569972 11126 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: No bootstrap required, opened a new log
I20260811 16:08:34.570465 11126 ts_tablet_manager.cc:1397] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
I20260811 16:08:34.571581 11130 raft_consensus.cc:348] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.571851 11127 raft_consensus.cc:348] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.572602 11130 raft_consensus.cc:374] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.572714 11127 raft_consensus.cc:374] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.573108 11130 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.573186 11127 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.573853 11126 raft_consensus.cc:348] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.574265 11127 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.574113 11130 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.574703 11126 raft_consensus.cc:374] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.575238 11126 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f50025c0d14d86a35172b2c6b84b2a, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.576000 11126 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.578711 11130 ts_tablet_manager.cc:1428] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
I20260811 16:08:34.581602 11130 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: Bootstrap starting.
I20260811 16:08:34.595268 11127 ts_tablet_manager.cc:1428] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: Time spent starting tablet: real 0.026s user 0.010s sys 0.002s
I20260811 16:08:34.596411 11127 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: Bootstrap starting.
I20260811 16:08:34.601047 11126 ts_tablet_manager.cc:1428] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Time spent starting tablet: real 0.030s user 0.015s sys 0.014s
I20260811 16:08:34.602010 11126 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: Bootstrap starting.
I20260811 16:08:34.604139 11130 tablet_bootstrap.cc:654] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.612980 11136 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:34.613763 11136 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.614953 11127 tablet_bootstrap.cc:654] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.619961 11126 tablet_bootstrap.cc:654] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.629238 11136 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.632561 11138 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:34.636488 11138 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.641752 11138 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:34.669853 11137 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:34.671090 11137 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.686851 11130 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: No bootstrap required, opened a new log
I20260811 16:08:34.687107 11137 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:34.687459 11130 ts_tablet_manager.cc:1397] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: Time spent bootstrapping tablet: real 0.106s user 0.017s sys 0.018s
I20260811 16:08:34.697875 11126 tablet_bootstrap.cc:492] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: No bootstrap required, opened a new log
I20260811 16:08:34.706969 11126 ts_tablet_manager.cc:1397] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: Time spent bootstrapping tablet: real 0.105s user 0.011s sys 0.013s
I20260811 16:08:34.734059 11126 raft_consensus.cc:348] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.734965 11126 raft_consensus.cc:374] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.735317 11126 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f50025c0d14d86a35172b2c6b84b2a, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.741917 11130 raft_consensus.cc:348] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.742925 11130 raft_consensus.cc:374] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.743264 11130 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.744086 11130 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.746851 11130 ts_tablet_manager.cc:1428] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: Time spent starting tablet: real 0.057s user 0.007s sys 0.000s
I20260811 16:08:34.747992 11130 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: Bootstrap starting.
I20260811 16:08:34.750193 11126 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.752708 11126 ts_tablet_manager.cc:1428] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: Time spent starting tablet: real 0.045s user 0.000s sys 0.009s
I20260811 16:08:34.753865 11126 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: Bootstrap starting.
I20260811 16:08:34.763684 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:34.766405 11127 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: No bootstrap required, opened a new log
I20260811 16:08:34.766948 11127 ts_tablet_manager.cc:1397] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: Time spent bootstrapping tablet: real 0.171s user 0.019s sys 0.041s
I20260811 16:08:34.771641 11126 tablet_bootstrap.cc:654] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: Neither blocks nor log segments found. Creating new log.
W20260811 16:08:34.778206 10695 leader_election.cc:341] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359): Illegal state: must be running to vote when last-logged opid is not known
I20260811 16:08:34.785897 11127 raft_consensus.cc:348] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.786710 11127 raft_consensus.cc:374] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.783607 11136 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:34.788688 11136 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.791033 11136 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.787161 11127 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.794644 11130 tablet_bootstrap.cc:654] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.797148 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
W20260811 16:08:34.803417 10696 leader_election.cc:341] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155): Illegal state: must be running to vote when last-logged opid is not known
I20260811 16:08:34.796413 11127 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.803452 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:34.805485 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:34.807109 10760 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 0.
I20260811 16:08:34.808682 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:34.809091 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:34.809475 10882 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 0.
I20260811 16:08:34.810470 10881 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 0.
W20260811 16:08:34.812755 10953 leader_election.cc:341] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591): Illegal state: must be running to vote when last-logged opid is not known
I20260811 16:08:34.814759 11127 ts_tablet_manager.cc:1428] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: Time spent starting tablet: real 0.047s user 0.007s sys 0.003s
I20260811 16:08:34.807801 10696 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:34.816572 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters: d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:34.817112 11138 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 16:08:34.822774 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters:
I20260811 16:08:34.821233 11136 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:34.825098 11136 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:34.825692 11136 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.826762 11126 tablet_bootstrap.cc:492] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: No bootstrap required, opened a new log
I20260811 16:08:34.827275 11126 ts_tablet_manager.cc:1397] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: Time spent bootstrapping tablet: real 0.074s user 0.004s sys 0.007s
I20260811 16:08:34.831429 11126 raft_consensus.cc:348] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.832233 11126 raft_consensus.cc:374] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.832540 11126 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f50025c0d14d86a35172b2c6b84b2a, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.832739 11136 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.835054 11164 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:34.835491 11164 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:34.835806 11164 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.836210 11136 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.839102 11127 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: Bootstrap starting.
I20260811 16:08:34.841886 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:34.839874 11126 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.850142 10760 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.852964 11130 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: No bootstrap required, opened a new log
I20260811 16:08:34.853515 11130 ts_tablet_manager.cc:1397] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: Time spent bootstrapping tablet: real 0.106s user 0.015s sys 0.001s
I20260811 16:08:34.855883 11126 ts_tablet_manager.cc:1428] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: Time spent starting tablet: real 0.028s user 0.007s sys 0.003s
I20260811 16:08:34.856503 11130 raft_consensus.cc:348] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.857313 11130 raft_consensus.cc:374] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.857594 11130 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.858268 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:34.858297 11130 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.861687 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:34.862473 10761 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 0.
I20260811 16:08:34.862797 11130 ts_tablet_manager.cc:1428] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260811 16:08:34.863823 10829 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:34.865372 10760 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 1.
I20260811 16:08:34.866050 11137 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:34.866550 11137 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:34.866891 11137 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.867769 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:34.868383 10881 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.876715 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:34.877867 11136 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:34.881067 10881 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 1.
I20260811 16:08:34.884980 11137 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.887748 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:34.887816 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:34.888610 10760 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.894515 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:34.895321 10760 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 1.
I20260811 16:08:34.896287 11127 tablet_bootstrap.cc:654] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: Neither blocks nor log segments found. Creating new log.
I20260811 16:08:34.896668 10829 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530
I20260811 16:08:34.898553 11169 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:34.899775 11164 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.901321 11012 raft_consensus.cc:2379] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ad56aab4d843c599224c82b3f41311 in current term 1: Already voted for candidate 429744fd0c424b86bef5550dccfc6530 in this term.
I20260811 16:08:34.902868 11137 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 1 election: Requested vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:34.905282 11136 raft_consensus.cc:686] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:34.906483 11136 consensus_queue.cc:237] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.907307 11164 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.908855 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:34.909335 11127 tablet_bootstrap.cc:492] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: No bootstrap required, opened a new log
I20260811 16:08:34.908869 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:34.909921 11127 ts_tablet_manager.cc:1397] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: Time spent bootstrapping tablet: real 0.071s user 0.014s sys 0.003s
I20260811 16:08:34.910424 10760 raft_consensus.cc:2379] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 429744fd0c424b86bef5550dccfc6530 in current term 1: Already voted for candidate 52ad56aab4d843c599224c82b3f41311 in this term.
I20260811 16:08:34.913123 11167 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:34.913805 11169 raft_consensus.cc:686] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:34.913734 11167 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.913915 11127 raft_consensus.cc:348] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.914855 11127 raft_consensus.cc:374] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:08:34.915266 11127 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Initialized, Role: FOLLOWER
I20260811 16:08:34.914773 11169 consensus_queue.cc:237] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:34.916472 11127 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.917590 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:34.918341 10760 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 0.
I20260811 16:08:34.919368 11127 ts_tablet_manager.cc:1428] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260811 16:08:34.921576 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:34.924819 11136 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:08:34.925783 11136 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:34.926100 11136 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.935839 11136 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.938256 11136 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.939174 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:34.939849 10760 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.942176 10432 catalog_manager.cc:5495] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 1 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:34.942831 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530; no voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:34.945464 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:34.945667 11136 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 16:08:34.946146 10881 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:08:34.946660 10760 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 1.
I20260811 16:08:34.948053 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:34.949220 11136 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:08:34.949759 11136 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:34.950649 11136 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:34.952072 10431 catalog_manager.cc:5495] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ad56aab4d843c599224c82b3f41311 (127.7.228.2). New cstate: current_term: 1 leader_uuid: "52ad56aab4d843c599224c82b3f41311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:34.956367 11167 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:34.956624 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:34.958860 10881 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 1.
I20260811 16:08:35.002717 11169 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:35.003316 11136 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 16:08:35.014523 10432 catalog_manager.cc:5495] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 1 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:35.065724 11169 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:35.096019 11164 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:08:35.097635 11164 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 16:08:35.109308 10380 debug-util.cc:398] Leaking SignalData structure 0x7b080004ab00 after lost signal to thread 8084
I20260811 16:08:35.136482 11164 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 16:08:35.200472 11138 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:35.201593 11138 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:35.204530 11138 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:35.210711 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:35.212321 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:35.217451 10696 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:35.218235 11138 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:35.592334 10881 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45"
dest_uuid: "52ad56aab4d843c599224c82b3f41311"
from {username='slave'} at 127.0.0.1:49230
I20260811 16:08:35.593132 10881 raft_consensus.cc:580] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 LEADER]: Received request to step down
I20260811 16:08:35.593441 10881 raft_consensus.cc:3032] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:35.593727 10881 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:35.594497 10881 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:35.595872 10881 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:35.701642 11167 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:35.702286 11167 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:35.704298 11167 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:35.705183 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:35.706581 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:35.707269 10882 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 2.
I20260811 16:08:35.708556 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters:
I20260811 16:08:35.709399 11167 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 16:08:35.709851 11167 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:35.710286 11167 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:35.717847 11167 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:35.720000 11167 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 2 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:35.720844 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:35.721963 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:35.729768 10882 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 2.
I20260811 16:08:35.731138 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters: d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:35.732379 11167 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:08:35.732909 11167 raft_consensus.cc:686] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:35.733748 11167 consensus_queue.cc:237] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:35.737589 11232 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:35.738050 11232 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:35.740183 11232 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:35.741006 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:35.741603 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:35.742406 10882 raft_consensus.cc:2379] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3f50025c0d14d86a35172b2c6b84b2a in current term 2: Already voted for candidate 429744fd0c424b86bef5550dccfc6530 in this term.
I20260811 16:08:35.743803 10695 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:35.744632 11232 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:35.750469 10426 catalog_manager.cc:5495] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 reported cstate change: term changed from 1 to 2, leader changed from 52ad56aab4d843c599224c82b3f41311 (127.7.228.2) to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 2 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:35.751637 11232 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:35.822139 10761 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:35.823907 11167 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:35.837085 10882 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:35.838840 11167 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:35.859401 11011 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb"
dest_uuid: "429744fd0c424b86bef5550dccfc6530"
from {username='slave'} at 127.0.0.1:47388
I20260811 16:08:35.860073 11011 raft_consensus.cc:580] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Received request to step down
I20260811 16:08:35.860407 11011 raft_consensus.cc:3032] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:35.860700 11011 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:35.861459 11011 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:35.862774 11011 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:35.972278 11172 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:35.972841 11172 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:35.974725 11172 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:35.977159 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:35.977918 11011 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
I20260811 16:08:35.983021 10830 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters:
I20260811 16:08:35.982723 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:35.984547 11172 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 16:08:35.984982 11172 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:35.985354 11172 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:35.990293 11172 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:35.992689 11172 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:35.996498 11232 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:35.997236 11232 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:35.999505 11232 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:36.001674 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:36.002823 10882 raft_consensus.cc:2379] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3f50025c0d14d86a35172b2c6b84b2a in current term 2: Already voted for candidate 52ad56aab4d843c599224c82b3f41311 in this term.
I20260811 16:08:36.096175 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:36.165508 11232 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:36.166224 11232 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:36.168437 11232 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:36.170643 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:36.173461 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:36.174257 10761 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:36.216475 11232 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:36.217286 11232 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:36.223062 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:36.241385 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:36.241987 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:36.297004 10696 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:36.304567 11244 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:36.305801 10696 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:36.306051 11011 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
W20260811 16:08:36.306684 11011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47376 (request call id 13) took 313 ms (client timeout 203 ms). Trace:
W20260811 16:08:36.307152 11011 rpcz_store.cc:269] 0811 16:08:35.992982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:36.095708 (+102726us) service_pool.cc:227] Handling call
0811 16:08:36.306618 (+210910us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 16:08:36.309283 11232 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:36.312731 11242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:08:36.317754 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
W20260811 16:08:36.319891 10830 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.228.3:37155, user_credentials={real_user=slave}} blocked reactor thread for 123582us
I20260811 16:08:36.321554 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:36.322301 11012 raft_consensus.cc:2362] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2. Re-sending same reply.
I20260811 16:08:36.332067 10830 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters: d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:36.335318 10761 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
W20260811 16:08:36.336066 10761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43040 (request call id 11) took 341 ms (client timeout 203 ms). Trace:
I20260811 16:08:36.313279 11242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:36.351361 11172 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:08:36.357342 11244 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:36.363827 11172 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:36.364965 11172 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:36.369644 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:36.370453 11242 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:36.372277 10696 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:36.373782 11232 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 16:08:36.336478 10761 rpcz_store.cc:269] 0811 16:08:35.994372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:36.173110 (+178738us) service_pool.cc:227] Handling call
0811 16:08:36.335991 (+162881us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 16:08:36.496069 11011 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:36.498023 11172 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:36.502354 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:36.503186 10341 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 2.
I20260811 16:08:36.503940 10761 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:36.504454 10511 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:36.505389 11242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 16:08:36.505800 11242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:08:36.505954 11172 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:36.506227 11242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:36.531986 11242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:36.638458 11242 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 3 election: Requested vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:36.661402 11232 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:36.662096 11232 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:36.664225 11232 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:36.666085 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:36.669312 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:36.671176 10695 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:36.672060 11232 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:36.692139 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc"
I20260811 16:08:36.692950 10341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 2 FOLLOWER]: Advancing to term 3
W20260811 16:08:36.705273 10953 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.2:40359, user_credentials={real_user=slave}} blocked reactor thread for 50614.9us
I20260811 16:08:36.727669 11253 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:36.728331 11253 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:36.751811 11253 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:36.752660 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:36.784014 10341 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 3.
I20260811 16:08:36.800210 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87"
I20260811 16:08:36.800611 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:36.801728 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:36.812170 10511 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc; no voters: 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:36.813097 11242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Leader election won for term 3
I20260811 16:08:36.813997 10341 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Rejecting Update request from peer 9f927cbce0704eac8a7d3836460dce87 for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:36.891788 11232 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:36.892482 11232 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:36.894713 11232 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:36.914516 11263 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: INVALID_TERM, Last received: 2.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260811 16:08:36.947639 11266 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:36.948138 11266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9f927cbce0704eac8a7d3836460dce87, State: Running, Role: LEADER
W20260811 16:08:36.949082 10412 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.62:43139, user_credentials={real_user=slave}} blocked reactor thread for 52681.8us
W20260811 16:08:37.081728 10511 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.228.62:43139, user_credentials={real_user=slave}} blocked reactor thread for 269648us
I20260811 16:08:37.149905 11266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:37.230798 11266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:37.338814 11267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:37.339473 11267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
W20260811 16:08:37.341845 11242 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.528s user 0.004s sys 0.010s
W20260811 16:08:37.342309 11242 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.529s user 0.004s sys 0.010s
I20260811 16:08:37.342619 11242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 LEADER]: Becoming Leader. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Running, Role: LEADER
I20260811 16:08:37.342785 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:37.344103 11269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:37.343719 11242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:37.344609 11269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:37.345657 10448 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17ce76946324f1d9576e7006ab317cc in term 3.
I20260811 16:08:37.487003 10550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:37.489374 11267 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:37.496158 10341 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Refusing update from remote peer 7050dea6bf96493a9a5ebfa411565d7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260811 16:08:37.510659 11271 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260811 16:08:37.612870 10309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.228.61:34181 timed out after 0.270s (SENT) [suppressed 10 similar messages]
W20260811 16:08:37.946352 10309 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.228.61:34181, user_credentials={real_user=slave}} blocked reactor thread for 333680us
I20260811 16:08:37.948387 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:37.950610 10309 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:37.951644 11267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260811 16:08:38.044859 10511 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.001s sys 0.012s
W20260811 16:08:38.045464 10511 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.547s user 0.001s sys 0.012s
W20260811 16:08:38.045881 10511 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.62:43139, user_credentials={real_user=slave}} blocked reactor thread for 547949us
W20260811 16:08:38.085903 10309 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.228.61:34181, user_credentials={real_user=slave}} blocked reactor thread for 135385us
I20260811 16:08:38.237852 10426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52222 (request call id 28) took 1287 ms. Trace:
I20260811 16:08:38.238483 10426 rpcz_store.cc:276] 0811 16:08:36.950308 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:37.083217 (+132909us) service_pool.cc:227] Handling call
0811 16:08:38.236678 (+1153461us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540004f6d4. stack: 00007fbc9bb2e033 00007fbc9bb2df30 00007fbc9b48c534 00000000003b4464 00000000003b43f9 00007fbca2352af0 00007fbca23d5f2a 00007fbca49eed6c 00007fbca4ae8c3e 00007fbca2df4fb4 00007fbca2df4f42 00007fbca2df4e71 00007fbca2df4dec 00007fbca2df40a2 00007fbca068e94c 00007fbca068dd86
0811 16:08:38.237753 (+ 1075us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":597610880}
I20260811 16:08:38.252174 11267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:38.252844 11267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:38.255079 11267 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:38.257169 10550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:38.260407 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:38.262205 10309 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:38.263724 11267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:38.349853 10552 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 LEADER]: Rejecting Update request from peer 9f927cbce0704eac8a7d3836460dce87 for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:38.350765 10552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41890 (request call id 27) took 1539 ms. Trace:
I20260811 16:08:38.351220 10552 rpcz_store.cc:276] 0811 16:08:36.811837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:38.349251 (+1537414us) service_pool.cc:227] Handling call
0811 16:08:38.349634 (+ 383us) raft_consensus.cc:1457] Updating replica for 0 ops
0811 16:08:38.350454 (+ 820us) raft_consensus.cc:1683] Filling consensus response to leader.
0811 16:08:38.350703 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 16:08:38.220785 10430 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52202 (request call id 26) took 2244 ms. Trace:
I20260811 16:08:38.378712 10430 rpcz_store.cc:276] 0811 16:08:35.976157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:35.976369 (+ 212us) service_pool.cc:227] Handling call
0811 16:08:38.220698 (+2244329us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":0}
I20260811 16:08:38.380026 11283 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:38.380991 11283 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:38.381922 11278 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:38.382326 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.383078 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:38.383270 11278 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.384292 11283 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
I20260811 16:08:38.387784 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:38.388516 10760 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
I20260811 16:08:38.389053 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:38.389735 10759 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 1.
I20260811 16:08:38.389886 10829 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:38.390779 11283 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 16:08:38.391206 11283 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:38.391120 10829 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:38.391710 11283 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:38.388478 11278 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:38.497884 11278 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 16:08:38.498431 11278 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:38.498921 11278 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:38.498945 11283 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:38.501320 11283 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 3 election: Requested vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
W20260811 16:08:38.505785 10829 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.228.1:34591, user_credentials={real_user=slave}} blocked reactor thread for 114730us
I20260811 16:08:38.507714 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:38.507537 11278 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.508358 10759 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:38.509702 11278 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:38.511569 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:38.512249 10760 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:38.515357 10759 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 3.
I20260811 16:08:38.516515 10829 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:38.517266 11278 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 16:08:38.517722 11278 raft_consensus.cc:686] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:38.518641 11278 consensus_queue.cc:237] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:38.520280 10760 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
I20260811 16:08:38.522341 10829 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:38.523201 11284 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:08:38.523639 11284 raft_consensus.cc:686] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:38.524341 11284 consensus_queue.cc:237] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
W20260811 16:08:38.528369 10953 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.739s user 0.009s sys 0.029s
W20260811 16:08:38.528975 10953 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.740s user 0.009s sys 0.029s
W20260811 16:08:38.531179 10953 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.2:40359, user_credentials={real_user=slave}} blocked reactor thread for 1.76418e+06us
W20260811 16:08:38.549185 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800042ea0 after lost signal to thread 8084
W20260811 16:08:38.549568 10830 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.857s user 0.010s sys 0.028s
W20260811 16:08:38.549973 10830 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.858s user 0.010s sys 0.028s
W20260811 16:08:38.550242 10830 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.3:37155, user_credentials={real_user=slave}} blocked reactor thread for 1.85788e+06us
I20260811 16:08:38.552883 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:38.554332 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:38.553375 11008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:38.563097 11244 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:38.563660 11244 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.564944 10759 raft_consensus.cc:1226] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 1. Current term is 2. Ops: []
I20260811 16:08:38.565824 11244 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:38.570858 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:38.575019 10882 raft_consensus.cc:1226] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 LEADER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:38.584146 11008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:38.585763 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:38.603847 11269 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:38.605753 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:38.606667 10760 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 2.
I20260811 16:08:38.609864 10450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Refusing update from remote peer 7050dea6bf96493a9a5ebfa411565d7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260811 16:08:38.610888 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:38.611773 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:38.613427 10411 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87; no voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc
W20260811 16:08:38.614166 10448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52162 (request call id 2) took 1272 ms (client timeout 270 ms). Trace:
I20260811 16:08:38.614719 11269 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260811 16:08:38.614563 10448 rpcz_store.cc:269] 0811 16:08:37.342343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:37.342508 (+ 165us) service_pool.cc:227] Handling call
0811 16:08:38.605176 (+1262668us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b540004f6d4. stack: 00007fbc9bb2e033 00007fbc9bb2df30 00007fbc9b48c534 00000000003b4464 00000000003b43f9 00007fbca2352471 00007fbca23d12ae 00007fbca45dac27 00007fbca1f62c04 00007fbca1f62b92 00007fbca1f62ac1 00007fbca1f62a3c 00007fbca1f61cf2 00007fbca068e94c 00007fbca068dd86 00007fbca068d72f
0811 16:08:38.614096 (+ 8920us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5039488}
I20260811 16:08:38.615952 11136 consensus_queue.cc:1046] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:38.618417 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:38.619145 10756 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 2.
I20260811 16:08:38.622483 10695 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:38.623364 11244 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 16:08:38.624586 11010 raft_consensus.cc:3032] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 16:08:38.624989 11010 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:38.625116 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:38.625849 11010 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.627032 11010 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:08:38.627276 10695 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:38.628386 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:38.630757 10650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 16:08:38.631695 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:38.632956 10756 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260811 16:08:38.634567 11284 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:38.637735 11264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.638568 11264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:38.663468 11136 consensus_queue.cc:1225] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Peer 52ad56aab4d843c599224c82b3f41311 log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260811 16:08:38.674897 10529 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41918 (request call id 29) took 2339 ms. Trace:
I20260811 16:08:38.675381 10529 rpcz_store.cc:276] 0811 16:08:36.329812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:36.329984 (+ 172us) service_pool.cc:227] Handling call
0811 16:08:38.667816 (+2337832us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b540020f654. stack: 00007fbc9bb2e033 00007fbc9bb2df30 00007fbc9b48c534 00000000003b4464 00000000003b43f9 00007fbca2352af0 00007fbca23d5f2a 00007fbca49eed6c 00007fbca4ae8c3e 00007fbca2df4fb4 00007fbca2df4f42 00007fbca2df4e71 00007fbca2df4dec 00007fbca2df40a2 00007fbca068e94c 00007fbca068dd86
0811 16:08:38.668788 (+ 972us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13623168}
W20260811 16:08:38.677842 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800060860 after lost signal to thread 10384
W20260811 16:08:38.678922 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08001586a0 after lost signal to thread 10482
I20260811 16:08:38.694996 10760 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:38.697829 11284 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:38.698832 11012 raft_consensus.cc:3032] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:38.699244 11012 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:38.700033 11012 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
W20260811 16:08:38.691057 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08001fc620 after lost signal to thread 10584
I20260811 16:08:38.711935 11012 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Advancing to term 3
W20260811 16:08:38.722453 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08001e1d40 after lost signal to thread 10801
I20260811 16:08:38.723510 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.724339 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:38.726970 11277 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260811 16:08:38.728793 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800225140 after lost signal to thread 10923
I20260811 16:08:38.728940 11302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:38.728291 11297 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260811 16:08:38.742599 10478 debug-util.cc:398] Leaking SignalData structure 0x7b080019f440 after lost signal to thread 11058
I20260811 16:08:38.742873 11010 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 16:08:38.745219 11297 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:38.745721 11284 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:08:38.757071 11271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.758026 11271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:38.758836 11244 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:38.759536 11012 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260811 16:08:38.753254 11297 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:38.752315 11302 catalog_manager.cc:669] Loaded metadata for table test-table [id=465c3f203d8b47ba84eb9663bfecfe23]
I20260811 16:08:38.775732 10758 raft_consensus.cc:1226] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 2. Current term is 3. Ops: []
W20260811 16:08:38.782557 11290 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260811 16:08:38.778723 10761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43040 (request call id 15) took 1962 ms. Trace:
I20260811 16:08:38.789036 11284 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:38.804845 11297 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.807113 11011 raft_consensus.cc:1226] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Rejecting Update request from peer 52ad56aab4d843c599224c82b3f41311 for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:38.790064 10761 rpcz_store.cc:276] 0811 16:08:36.815397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 16:08:38.623309 (+1807912us) service_pool.cc:227] Handling call
0811 16:08:38.623691 (+ 382us) raft_consensus.cc:1457] Updating replica for 0 ops
0811 16:08:38.623884 (+ 193us) raft_consensus.cc:1520] Early marking committed up to index 3
0811 16:08:38.680995 (+ 57111us) log.cc:842] Serialized 14 byte log entry
0811 16:08:38.740416 (+ 59421us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0811 16:08:38.753106 (+ 12690us) raft_consensus.cc:1631] Marking committed up to 3
0811 16:08:38.753959 (+ 853us) raft_consensus.cc:1683] Filling consensus response to leader.
0811 16:08:38.773385 (+ 19426us) raft_consensus.cc:1677] UpdateReplicas() finished
0811 16:08:38.778017 (+ 4632us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":51610,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":22819,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":198,"thread_start_us":98674,"threads_started":2,"wal-append.queue_time_us":101098}
I20260811 16:08:38.810963 11284 consensus_queue.cc:1046] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260811 16:08:38.821897 11278 raft_consensus.cc:3032] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:38.822381 11278 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:38.823223 11278 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.827051 11278 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:38.828722 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:38.829484 10761 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:38.841369 10761 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 3.
I20260811 16:08:38.841317 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:38.841526 11297 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:38.850314 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:38.851308 11290 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 16:08:38.851790 11290 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:38.852564 11290 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:38.854117 10882 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 3.
I20260811 16:08:38.846310 11302 tablet_loader.cc:96] loaded metadata for tablet 6afe0fe649d34850a77793d9372f92fb (table test-table [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:08:38.863214 11302 tablet_loader.cc:96] loaded metadata for tablet 84e71aac7b9b46479a8ce87b88b74e45 (table test-table [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:08:38.864588 11302 tablet_loader.cc:96] loaded metadata for tablet d42b5e0273654c95badc4250fc283cbd (table test-table [id=465c3f203d8b47ba84eb9663bfecfe23])
W20260811 16:08:38.865548 11312 log.cc:924] Time spent T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260811 16:08:38.867228 11302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:38.874023 11302 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:38.874526 11302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:38.880342 11302 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:38.926463 11302 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Loaded TSK: 0
I20260811 16:08:38.929334 11302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:38.933275 11269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.934675 11269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:38.949430 11269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:38.950296 11269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:38.953253 11269 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:38.955715 10552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:38.960228 11322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.966626 11322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:38.956382 11277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:38.967864 11277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:38.978349 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:38.984469 10760 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260811 16:08:38.986770 10882 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260811 16:08:38.986579 11297 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260811 16:08:38.987449 10532 catalog_manager.cc:5495] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 reported cstate change: term changed from 2 to 3, leader changed from 429744fd0c424b86bef5550dccfc6530 (127.7.228.3) to 52ad56aab4d843c599224c82b3f41311 (127.7.228.2). New cstate: current_term: 3 leader_uuid: "52ad56aab4d843c599224c82b3f41311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:38.989477 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87; no voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc
I20260811 16:08:38.990579 11322 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:38.991541 11297 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 16:08:38.986084 10533 catalog_manager.cc:5495] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 reported cstate change: term changed from 1 to 2, leader changed from 429744fd0c424b86bef5550dccfc6530 (127.7.228.3) to 52ad56aab4d843c599224c82b3f41311 (127.7.228.2). New cstate: current_term: 2 leader_uuid: "52ad56aab4d843c599224c82b3f41311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } }
I20260811 16:08:39.017021 10529 catalog_manager.cc:5495] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } }
I20260811 16:08:39.082108 10925 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:39.099983 11066 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:39.131084 10802 heartbeater.cc:503] Master 127.7.228.60:38613 was elected leader, sending a full tablet report...
I20260811 16:08:39.248395 10533 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56982:
table { table_id: "465c3f203d8b47ba84eb9663bfecfe23" table_name: "test-table" } new_table_name: "NewTableName"
I20260811 16:08:39.387950 8080 master.cc:537] Master@127.7.228.60:38613 shutting down...
I20260811 16:08:39.538684 11347 tablet.cc:1722] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.541862 11351 tablet.cc:1722] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.547716 11349 tablet.cc:1722] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.540760 11350 tablet.cc:1722] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.555502 11354 tablet.cc:1722] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.558454 11358 tablet.cc:1722] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.646274 11348 tablet.cc:1722] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.647957 11353 tablet.cc:1722] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.663905 11356 tablet.cc:1722] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 16:08:39.730922 11359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:39.731586 11359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:39.735877 11359 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:39.772634 11360 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:39.773308 11360 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:39.774314 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:39.775048 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f927cbce0704eac8a7d3836460dce87 in term 3.
I20260811 16:08:39.776038 11360 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:39.779258 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:39.781402 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
W20260811 16:08:39.789754 10305 leader_election.cc:334] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 16:08:39.791246 10309 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:39.791488 11359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 16:08:39.791893 11359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7050dea6bf96493a9a5ebfa411565d7f)
I20260811 16:08:39.792233 11359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:39.792366 11360 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:39.805606 11359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:39.810322 10342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f927cbce0704eac8a7d3836460dce87" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc"
I20260811 16:08:39.810890 11359 leader_election.cc:288] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 election: Requested vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:39.810994 10342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:39.818322 10342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f927cbce0704eac8a7d3836460dce87 in term 4.
I20260811 16:08:39.819793 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:08:39.819869 10341 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 3. Current term is 4. Ops: []
I20260811 16:08:39.825788 11359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 16:08:39.838102 11359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 LEADER]: Becoming Leader. State: Replica: 9f927cbce0704eac8a7d3836460dce87, State: Running, Role: LEADER
I20260811 16:08:39.841866 11359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
W20260811 16:08:39.840835 10411 leader_election.cc:334] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 16:08:39.844528 10411 leader_election.cc:334] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 16:08:39.849661 11370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:39.850466 11284 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:39.850651 11370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:39.850924 10450 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 LEADER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 3. Current term is 4. Ops: []
I20260811 16:08:39.852811 11271 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 }, Status: INVALID_TERM, Last received: 3.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260811 16:08:39.856082 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 LEADER]: Raft consensus shutting down.
I20260811 16:08:39.851042 11284 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:39.869485 11284 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:39.871560 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:39.873235 11271 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer 9f927cbce0704eac8a7d3836460dce87 log is divergent from this leader: its last log entry 4.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260811 16:08:39.947733 11375 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: INVALID_TERM, Last received: 3.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260811 16:08:39.955519 10411 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 -> Peer 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613): Couldn't send request to peer 7050dea6bf96493a9a5ebfa411565d7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:08:39.959003 10341 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Refusing update from remote peer 9f927cbce0704eac8a7d3836460dce87: Log matching property violated. Preceding OpId in replica: term: 3 index: 16. Preceding OpId from leader: term: 4 index: 17. (index mismatch)
I20260811 16:08:39.961458 11370 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260811 16:08:39.965301 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:39.966143 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:39.877923 11373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:08:39.905906 11371 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:39.969975 11371 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
W20260811 16:08:39.973191 11374 raft_consensus.cc:918] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 LEADER]: Unable to handle notification of new term (4): Illegal state: RaftConsensus is not running: State = Stopping
I20260811 16:08:39.974960 11371 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:39.975755 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 16:08:39.976431 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: stopping tablet replica
I20260811 16:08:39.987227 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:39.989308 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:39.993137 10695 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:39.994253 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:39.995141 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:39.995543 11371 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:40.006990 11370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:40.007867 11370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:40.009891 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:40.011898 10830 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311; no voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:40.012574 11373 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23]
I20260811 16:08:40.013464 11283 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:40.027366 11373 tablet_loader.cc:96] loaded metadata for tablet 6afe0fe649d34850a77793d9372f92fb (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:08:40.033353 11373 tablet_loader.cc:96] loaded metadata for tablet 84e71aac7b9b46479a8ce87b88b74e45 (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:08:40.037038 8080 master.cc:552] Master@127.7.228.60:38613 shutdown complete.
I20260811 16:08:40.038168 11373 tablet_loader.cc:96] loaded metadata for tablet d42b5e0273654c95badc4250fc283cbd (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:08:40.041152 11373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:08:40.056317 11373 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:40.060552 11373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:08:40.068203 11373 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:08:40.073887 11373 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: Loaded TSK: 0
I20260811 16:08:40.075650 11373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:08:40.214942 10803 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:40.242403 11067 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:40.258658 10430 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23] alter complete (version 1)
I20260811 16:08:40.269850 10929 heartbeater.cc:503] Master 127.7.228.61:34181 was elected leader, sending a full tablet report...
I20260811 16:08:40.277945 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:08:40.289464 11391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:40.290787 11392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:08:40.307070 11398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:08:40.307174 8080 server_base.cc:1034] running on GCE node
I20260811 16:08:40.309132 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:08:40.309486 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:08:40.309701 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464520309686 us; error 0 us; skew 500 ppm
I20260811 16:08:40.310446 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:08:40.336434 8080 webserver.cc:460] Webserver started at http://127.7.228.60:43717/ using document root <none> and password file <none>
I20260811 16:08:40.337620 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:08:40.337947 8080 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260811 16:08:40.349994 8080 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.009s sys 0.001s
I20260811 16:08:40.361341 11407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:08:40.362464 8080 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.009s
I20260811 16:08:40.362864 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
uuid: "7050dea6bf96493a9a5ebfa411565d7f"
format_stamp: "Formatted at 2026-08-11 16:08:33 on dist-test-slave-ncp9"
I20260811 16:08:40.363401 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:08:40.396246 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:08:40.398239 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260811 16:08:40.453722 10411 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 -> Peer 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613): Couldn't send request to peer 7050dea6bf96493a9a5ebfa411565d7f. Status: Network error: Client connection negotiation failed: client connection to 127.7.228.60:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 16:08:40.611825 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.60:38613
I20260811 16:08:40.611949 11485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.60:38613 every 8 connection(s)
I20260811 16:08:40.638890 11486 sys_catalog.cc:263] Verifying existing consensus state
I20260811 16:08:40.647228 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Bootstrap starting.
I20260811 16:08:40.712265 10802 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:40.776396 10925 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:40.784754 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=7 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:08:40.798451 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Bootstrap complete.
I20260811 16:08:40.806171 11486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:40.808809 11486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Initialized, Role: FOLLOWER
I20260811 16:08:40.810544 11486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:40.820230 11066 heartbeater.cc:347] Connected to a master server at 127.7.228.60:38613
I20260811 16:08:40.825843 11495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:40.826754 11495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:40.827654 11486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:08:40.828184 11449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:40.881927 11504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:40.882843 11504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:08:40.883611 11495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f927cbce0704eac8a7d3836460dce87. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:40.884402 11495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
W20260811 16:08:40.891351 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08001da060 after lost signal to thread 8084
W20260811 16:08:40.892719 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08001da7c0 after lost signal to thread 10384
W20260811 16:08:40.893846 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08001dae80 after lost signal to thread 10482
W20260811 16:08:40.895377 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08002339c0 after lost signal to thread 10801
W20260811 16:08:40.896219 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08000cdbc0 after lost signal to thread 10923
W20260811 16:08:40.897136 10919 debug-util.cc:398] Leaking SignalData structure 0x7b08000abf60 after lost signal to thread 11058
I20260811 16:08:40.904264 11529 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:08:40.904696 11529 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: loading cluster ID for follower catalog manager: success
I20260811 16:08:40.941221 11435 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "429744fd0c424b86bef5550dccfc6530" instance_seqno: 1786464513788962) as {username='slave'} at 127.0.0.1:57276; Asking this server to re-register.
I20260811 16:08:40.938283 11436 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "52ad56aab4d843c599224c82b3f41311" instance_seqno: 1786464513622423) as {username='slave'} at 127.0.0.1:57282; Asking this server to re-register.
I20260811 16:08:40.945830 11066 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:40.946642 11529 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: acquiring CA information for follower catalog manager: success
I20260811 16:08:40.946936 10925 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:40.950793 11435 ts_manager.cc:194] Registered new tserver with Master: 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:40.953527 11436 ts_manager.cc:194] Registered new tserver with Master: 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:40.960039 11529 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 16:08:40.967711 11370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "9f927cbce0704eac8a7d3836460dce87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:08:40.970340 11370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: LEADER
I20260811 16:08:40.972488 11436 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" instance_seqno: 1786464513475722) as {username='slave'} at 127.0.0.1:57256; Asking this server to re-register.
I20260811 16:08:40.980185 10802 heartbeater.cc:464] Registering TS with master...
I20260811 16:08:40.984380 11436 ts_manager.cc:194] Registered new tserver with Master: d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591)
W20260811 16:08:42.371346 10953 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.2:40359, user_credentials={real_user=slave}} blocked reactor thread for 72496.2us
W20260811 16:08:42.510383 10953 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.2:40359, user_credentials={real_user=slave}} blocked reactor thread for 53773.8us
W20260811 16:08:43.089530 10829 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.228.1:34591, user_credentials={real_user=slave}} blocked reactor thread for 55315.3us
I20260811 16:08:55.385885 11579 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:55.386873 11579 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.390235 11579 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:55.395159 11579 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:55.395846 11579 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.398484 11579 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:55.418273 11579 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:55.418931 11579 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:55.421195 11579 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
W20260811 16:08:55.604739 10695 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.228.2:40359 timed out after 0.214s (SENT) [suppressed 11 similar messages]
I20260811 16:08:55.606178 11570 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:55.606747 11570 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.607296 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.608467 10881 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f50025c0d14d86a35172b2c6b84b2a in term 3.
I20260811 16:08:55.607646 10877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.609824 11570 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:55.610795 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.616307 11572 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:55.616998 11572 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:55.618906 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:55.619587 10761 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 3.
I20260811 16:08:55.621263 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:55.621938 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:55.623588 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:55.620029 11572 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:55.625528 10829 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530
I20260811 16:08:55.625733 11572 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:55.626479 11582 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 16:08:55.626302 11572 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.626858 11582 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:55.627264 11582 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:55.628472 11572 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:55.636483 10695 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530
I20260811 16:08:55.636451 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.637573 11579 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 16:08:55.638010 11579 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:55.638549 11579 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:55.641073 11582 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.643230 11582 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:55.644548 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:55.644974 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:55.645659 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:55.644583 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:55.646579 11579 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.647550 11585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:08:55.648077 11585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:55.638021 10695 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:55.649842 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:55.648020 10761 raft_consensus.cc:2379] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ad56aab4d843c599224c82b3f41311 in current term 4: Already voted for candidate d3f50025c0d14d86a35172b2c6b84b2a in this term.
I20260811 16:08:55.648660 11579 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 election: Requested vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:55.652055 11585 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:55.652618 11579 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:55.653755 10829 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311; no voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:55.654914 11582 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260811 16:08:55.659125 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.659894 10695 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:55.661233 11588 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:55.660633 10877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.662324 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:55.662860 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:55.663467 10756 raft_consensus.cc:1226] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 3. Current term is 4. Ops: []
I20260811 16:08:55.664314 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:55.662211 10881 raft_consensus.cc:1226] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 3. Current term is 4. Ops: []
I20260811 16:08:55.665887 10695 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:55.666435 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530; no voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:55.667204 11572 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:55.668020 11583 consensus_queue.cc:1046] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260811 16:08:55.669592 11587 raft_consensus.cc:3032] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 16:08:55.672472 11587 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:55.671226 11588 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260811 16:08:55.667981 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530; no voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:55.674028 11572 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:55.673828 11587 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.674908 11572 consensus_queue.cc:1046] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 16:08:55.678498 11587 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:55.698608 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:55.703826 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:55.705943 11409 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f; no voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc
I20260811 16:08:55.706907 11585 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 16:08:55.888933 11587 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:55.889417 11587 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.891188 11587 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:55.891947 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:55.892125 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:55.892572 10756 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 4.
I20260811 16:08:55.892745 10881 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 4.
I20260811 16:08:55.893637 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:55.894299 11587 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 16:08:55.894649 11587 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:55.894927 11587 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:08:55.900804 11587 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.902855 11587 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:55.903497 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:55.903712 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:55.904028 10756 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:08:55.904274 10881 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:08:55.909792 10756 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 5.
I20260811 16:08:55.910364 10881 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 5.
I20260811 16:08:55.910924 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:55.911588 11587 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Leader election won for term 5
I20260811 16:08:55.912096 11587 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:55.914187 11587 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:55.925321 10430 catalog_manager.cc:5495] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: HEALTHY } } }
I20260811 16:08:56.031427 10756 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260811 16:08:56.032992 11587 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 16:08:56.035847 10881 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260811 16:08:56.037313 11583 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 16:08:56.306231 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800112140 after lost signal to thread 8084
W20260811 16:08:56.307276 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800225280 after lost signal to thread 10384
W20260811 16:08:56.308125 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800224fe0 after lost signal to thread 10482
W20260811 16:08:56.309438 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800118700 after lost signal to thread 10801
W20260811 16:08:56.310334 10478 debug-util.cc:398] Leaking SignalData structure 0x7b080008c000 after lost signal to thread 10923
W20260811 16:08:56.311219 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8f00 after lost signal to thread 11058
W20260811 16:08:56.311995 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800206900 after lost signal to thread 11485
I20260811 16:08:57.900952 11614 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:08:57.901592 11614 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:57.904381 11614 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:08:57.905562 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 19 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:57.927103 11583 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:57.927729 11583 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:57.929921 11583 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:57.965833 11617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:08:57.967780 11617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:08:57.969594 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:57.971832 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 19 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:08:57.973519 11619 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:57.974587 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 19 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:08:57.981592 11619 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:57.982964 11409 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f; no voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc
I20260811 16:08:57.984553 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:57.973855 11617 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:08:57.985900 11617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 16:08:57.987347 11623 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:57.987915 11623 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:08:57.990765 11623 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:57.991047 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:57.991358 11623 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:57.994143 11627 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:57.995939 11627 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:57.996484 11619 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:57.998962 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:57.999147 11627 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:57.999465 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:58.000155 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:58.000602 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:58.000159 10756 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 5.
I20260811 16:08:58.003043 10829 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530
I20260811 16:08:58.003926 10695 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:58.004320 11582 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260811 16:08:58.004745 11582 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 429744fd0c424b86bef5550dccfc6530)
I20260811 16:08:58.004918 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 19 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:08:58.005165 11582 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:08:58.005838 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:58.006778 10305 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:08:58.007313 10695 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:58.007980 11614 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 16:08:58.009074 11627 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:08:57.991824 11623 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.009939 11627 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260811 16:08:58.010248 10760 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 2.
I20260811 16:08:58.013163 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:58.013206 11582 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.014232 11583 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 16:08:58.014601 11583 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:08:58.014925 11583 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:58.015712 11582 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 6 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:58.016076 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:58.016319 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:58.016559 10761 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:08:58.020557 11583 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.021073 10761 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 6.
I20260811 16:08:58.022053 10829 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530
I20260811 16:08:58.023157 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:08:58.023507 11582 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 FOLLOWER]: Leader election won for term 6
I20260811 16:08:58.023695 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:58.024227 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:08:58.024725 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:58.025282 11582 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:58.025367 11623 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:58.023345 11583 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:08:58.026552 10695 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:08:58.026273 11582 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.026554 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530; no voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:08:58.027581 11583 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260811 16:08:58.027905 11623 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 16:08:58.036708 10430 catalog_manager.cc:5495] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 reported cstate change: term changed from 5 to 6, leader changed from 429744fd0c424b86bef5550dccfc6530 (127.7.228.3) to 52ad56aab4d843c599224c82b3f41311 (127.7.228.2). New cstate: current_term: 6 leader_uuid: "52ad56aab4d843c599224c82b3f41311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:58.045832 10881 raft_consensus.cc:1226] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 LEADER]: Rejecting Update request from peer 429744fd0c424b86bef5550dccfc6530 for earlier term 5. Current term is 6. Ops: []
I20260811 16:08:58.049844 11012 raft_consensus.cc:1226] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Rejecting Update request from peer 52ad56aab4d843c599224c82b3f41311 for earlier term 2. Current term is 3. Ops: []
I20260811 16:08:58.054235 11582 consensus_queue.cc:1046] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260811 16:08:58.055521 11619 raft_consensus.cc:3032] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 16:08:58.055982 11619 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:58.054234 11583 consensus_queue.cc:1046] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260811 16:08:58.056636 11583 consensus_queue.cc:1225] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Peer 52ad56aab4d843c599224c82b3f41311 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260811 16:08:58.057314 11616 raft_consensus.cc:3032] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 LEADER]: Stepping down as leader of term 5
I20260811 16:08:58.057689 11616 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:08:58.056872 11619 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.058606 11619 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 2 FOLLOWER]: Advancing to term 3
I20260811 16:08:58.058454 11616 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.059862 11616 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:08:58.148666 11012 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 6 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260811 16:08:58.148783 10761 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260811 16:08:58.150013 11582 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 16:08:58.150626 11619 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 16:08:58.234455 11570 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:08:58.234884 11570 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.236389 11570 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:58.237244 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:08:58.237387 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:08:58.237767 10761 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 2.
I20260811 16:08:58.237931 11012 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 3.
I20260811 16:08:58.238633 10829 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:58.239238 11570 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 16:08:58.239488 11570 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:08:58.239730 11570 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:58.245640 11570 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.247251 11570 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:08:58.248167 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:08:58.248172 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "52ad56aab4d843c599224c82b3f41311" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:08:58.248944 11012 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:08:58.249046 10761 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 2 FOLLOWER]: Advancing to term 4
I20260811 16:08:58.255666 11012 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 4.
I20260811 16:08:58.255662 10761 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ad56aab4d843c599224c82b3f41311 in term 4.
I20260811 16:08:58.256928 10829 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:08:58.257594 11570 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 16:08:58.257988 11570 raft_consensus.cc:686] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 LEADER]: Becoming Leader. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:08:58.258612 11570 consensus_queue.cc:237] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:08:58.266845 10429 catalog_manager.cc:5495] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "52ad56aab4d843c599224c82b3f41311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } health_report { overall_health: UNKNOWN } } }
I20260811 16:08:58.383229 11012 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260811 16:08:58.383776 10761 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Refusing update from remote peer 52ad56aab4d843c599224c82b3f41311: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260811 16:08:58.385897 11619 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 16:08:58.386942 11582 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260811 16:08:58.820744 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800206b80 after lost signal to thread 8084
W20260811 16:08:58.821889 10478 debug-util.cc:398] Leaking SignalData structure 0x7b080022cce0 after lost signal to thread 10384
W20260811 16:08:58.822772 10478 debug-util.cc:398] Leaking SignalData structure 0x7b080025d200 after lost signal to thread 10482
W20260811 16:08:58.823994 10478 debug-util.cc:398] Leaking SignalData structure 0x7b0800199a60 after lost signal to thread 10801
W20260811 16:08:58.824932 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08001e7d80 after lost signal to thread 10923
W20260811 16:08:58.825990 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3a40 after lost signal to thread 11058
W20260811 16:08:58.826815 10478 debug-util.cc:398] Leaking SignalData structure 0x7b08000f55a0 after lost signal to thread 11485
I20260811 16:09:02.670411 11676 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.671609 11676 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:09:02.709872 11674 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:09:02.710613 11674 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:02.818287 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:09:02.821638 11676 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:02.822333 11676 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.822876 11676 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.823248 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.827013 11674 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:09:02.828190 11677 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.828773 11677 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.831043 11677 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:09:02.838063 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.824880 11676 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:09:02.890959 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.903477 11683 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.904122 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:09:02.904274 11683 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.910331 11683 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:02.911074 11683 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.904846 11449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17ce76946324f1d9576e7006ab317cc in term 4.
I20260811 16:09:02.911598 11683 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.913471 10305 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, a17ce76946324f1d9576e7006ab317cc; no voters: 9f927cbce0704eac8a7d3836460dce87
I20260811 16:09:02.913947 11683 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:02.914701 11674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 16:09:02.915120 11674 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:09:02.915455 11674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 4 FOLLOWER]: Advancing to term 5
W20260811 16:09:02.917925 10411 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.693s user 0.012s sys 0.010s
W20260811 16:09:02.918323 10411 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.693s user 0.012s sys 0.010s
I20260811 16:09:02.919093 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:09:02.919749 11011 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f50025c0d14d86a35172b2c6b84b2a in term 6.
I20260811 16:09:02.920573 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:09:02.921212 11012 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f50025c0d14d86a35172b2c6b84b2a in term 4.
I20260811 16:09:02.907994 11684 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.925096 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:09:02.925398 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.925320 11684 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:09:02.926978 10696 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.928247 10696 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.929208 11682 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260811 16:09:02.929639 11682 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.929973 11682 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 6 FOLLOWER]: Advancing to term 7
I20260811 16:09:02.931006 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:09:02.931574 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.931648 10761 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 4.
I20260811 16:09:02.933830 11676 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 16:09:02.934224 11676 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.934577 11676 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:09:02.936939 11684 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:02.939884 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530" is_pre_election: true
I20260811 16:09:02.940467 11674 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:02.944407 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87"
I20260811 16:09:02.946790 10341 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Rejecting Update request from peer 9f927cbce0704eac8a7d3836460dce87 for earlier term 4. Current term is 5. Ops: []
I20260811 16:09:02.947520 10696 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.948371 11677 raft_consensus.cc:2726] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 16:09:02.948547 11668 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: INVALID_TERM, Last received: 4.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260811 16:09:02.949769 11668 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 LEADER]: Stepping down as leader of term 4
I20260811 16:09:02.950102 11668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9f927cbce0704eac8a7d3836460dce87, State: Running, Role: LEADER
I20260811 16:09:02.950970 11668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 4.21, Last appended by leader: 21, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:02.952246 11668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:09:02.955070 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f"
I20260811 16:09:02.956211 11676 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.956655 10305 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a17ce76946324f1d9576e7006ab317cc; no voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87
I20260811 16:09:02.958331 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:02.958134 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:09:02.950940 11682 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.959568 10761 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 3.
I20260811 16:09:02.957991 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:09:02.961323 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:09:02.961884 11676 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 5 election: Requested vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:09:02.963260 11684 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 16:09:02.960793 10760 raft_consensus.cc:2379] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 429744fd0c424b86bef5550dccfc6530 in current term 7: Already voted for candidate d3f50025c0d14d86a35172b2c6b84b2a in this term.
I20260811 16:09:02.963867 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:09:02.963646 11684 raft_consensus.cc:482] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.964658 11682 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 7 election: Requested vote from peers 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359), 429744fd0c424b86bef5550dccfc6530 (127.7.228.3:37155)
I20260811 16:09:02.964947 11684 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:09:02.963683 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.966718 11683 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 16:09:02.967072 11683 raft_consensus.cc:482] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ad56aab4d843c599224c82b3f41311)
I20260811 16:09:02.967481 11683 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:09:02.967613 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "429744fd0c424b86bef5550dccfc6530"
I20260811 16:09:02.968204 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:09:02.967638 10696 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.957165 11674 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 5 election: Requested vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:09:02.973196 10696 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3f50025c0d14d86a35172b2c6b84b2a; no voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.973048 11674 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260811 16:09:02.974363 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530; no voters: 52ad56aab4d843c599224c82b3f41311, d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:09:02.975065 11682 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260811 16:09:02.974555 11676 raft_consensus.cc:2726] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260811 16:09:02.978626 11683 raft_consensus.cc:504] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:09:02.980080 11010 raft_consensus.cc:1226] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Rejecting Update request from peer 52ad56aab4d843c599224c82b3f41311 for earlier term 3. Current term is 4. Ops: []
I20260811 16:09:02.980619 10760 raft_consensus.cc:1226] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Rejecting Update request from peer 52ad56aab4d843c599224c82b3f41311 for earlier term 4. Current term is 5. Ops: []
I20260811 16:09:02.980770 11685 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 6 FOLLOWER]: Advancing to term 7
I20260811 16:09:02.982754 11671 consensus_queue.cc:1046] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260811 16:09:02.983858 11666 consensus_queue.cc:1046] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260811 16:09:02.984510 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:09:02.984937 11666 raft_consensus.cc:3032] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 LEADER]: Stepping down as leader of term 4
I20260811 16:09:02.985438 10760 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:09:02.985273 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84e71aac7b9b46479a8ce87b88b74e45" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:09:02.985774 11680 raft_consensus.cc:3032] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 16:09:02.986066 11683 leader_election.cc:288] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:02.985296 11666 raft_consensus.cc:729] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:09:02.987231 11666 consensus_queue.cc:260] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:02.986116 11680 raft_consensus.cc:729] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:09:02.988711 11680 consensus_queue.cc:260] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:09:02.989907 11685 raft_consensus.cc:2726] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 16:09:02.993172 10760 raft_consensus.cc:2454] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 4.
I20260811 16:09:02.994414 10953 leader_election.cc:302] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters: 52ad56aab4d843c599224c82b3f41311
I20260811 16:09:02.989960 11680 raft_consensus.cc:3037] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 3 FOLLOWER]: Advancing to term 4
I20260811 16:09:02.997519 11685 raft_consensus.cc:2781] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 16:09:02.998834 10760 raft_consensus.cc:1226] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Rejecting Update request from peer 52ad56aab4d843c599224c82b3f41311 for earlier term 6. Current term is 7. Ops: []
I20260811 16:09:02.998874 11685 raft_consensus.cc:686] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:09:03.000735 11671 consensus_queue.cc:1046] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260811 16:09:03.000610 11685 consensus_queue.cc:237] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } }
I20260811 16:09:03.002866 11684 raft_consensus.cc:504] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.004065 11671 raft_consensus.cc:3032] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 LEADER]: Stepping down as leader of term 6
I20260811 16:09:02.995289 11666 raft_consensus.cc:3037] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Advancing to term 5
I20260811 16:09:03.004560 11671 raft_consensus.cc:729] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ad56aab4d843c599224c82b3f41311, State: Running, Role: LEADER
I20260811 16:09:03.006048 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:09:03.006788 10760 raft_consensus.cc:2379] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 429744fd0c424b86bef5550dccfc6530 in current term 5: Already voted for candidate d3f50025c0d14d86a35172b2c6b84b2a in this term.
I20260811 16:09:03.006713 11684 leader_election.cc:288] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:03.007359 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42b5e0273654c95badc4250fc283cbd" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:09:03.008128 11671 consensus_queue.cc:260] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.009469 11671 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 6 FOLLOWER]: Advancing to term 7
I20260811 16:09:03.022330 10882 raft_consensus.cc:2454] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 5.
I20260811 16:09:03.023907 10953 leader_election.cc:302] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, 52ad56aab4d843c599224c82b3f41311; no voters: d3f50025c0d14d86a35172b2c6b84b2a
I20260811 16:09:03.024578 11685 raft_consensus.cc:2781] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Leader election won for term 5
I20260811 16:09:03.025087 11685 raft_consensus.cc:686] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 5 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:09:03.025949 11685 consensus_queue.cc:237] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.110220 10882 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 16:09:03.112007 11685 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 16:09:03.117763 10760 raft_consensus.cc:1261] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 16:09:03.119377 11684 consensus_queue.cc:1035] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 16:09:03.125888 10761 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260811 16:09:03.127735 11700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:09:03.128312 11700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.131287 10881 raft_consensus.cc:1261] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260811 16:09:03.132503 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc" is_pre_election: true
I20260811 16:09:03.133127 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:09:03.133849 10450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 5.
I20260811 16:09:03.133399 10341 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7050dea6bf96493a9a5ebfa411565d7f in current term 5: Already voted for candidate a17ce76946324f1d9576e7006ab317cc in this term.
I20260811 16:09:03.136464 11409 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87; no voters: a17ce76946324f1d9576e7006ab317cc
W20260811 16:09:03.139041 10796 debug-util.cc:398] Leaking SignalData structure 0x7b0800198ac0 after lost signal to thread 8084
I20260811 16:09:03.139590 11683 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260811 16:09:03.140218 10796 debug-util.cc:398] Leaking SignalData structure 0x7b08001a6cc0 after lost signal to thread 10384
W20260811 16:09:03.141252 10796 debug-util.cc:398] Leaking SignalData structure 0x7b08001a70a0 after lost signal to thread 10482
W20260811 16:09:03.142731 10796 debug-util.cc:398] Leaking SignalData structure 0x7b0800064620 after lost signal to thread 10801
W20260811 16:09:03.144502 10796 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8ee0 after lost signal to thread 10923
I20260811 16:09:03.145817 11703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 16:09:03.142798 11683 consensus_queue.cc:1035] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 16:09:03.145848 10796 debug-util.cc:398] Leaking SignalData structure 0x7b08000f4c00 after lost signal to thread 11058
I20260811 16:09:03.146327 11703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9f927cbce0704eac8a7d3836460dce87)
I20260811 16:09:03.147893 11703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 4 FOLLOWER]: Advancing to term 5
W20260811 16:09:03.148293 10796 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf400 after lost signal to thread 11485
I20260811 16:09:03.150270 11700 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:09:03.168210 11703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.170644 11703 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 election: Requested vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
I20260811 16:09:03.171765 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "a17ce76946324f1d9576e7006ab317cc"
I20260811 16:09:03.172606 10341 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7050dea6bf96493a9a5ebfa411565d7f in current term 5: Already voted for candidate a17ce76946324f1d9576e7006ab317cc in this term.
I20260811 16:09:03.175040 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7050dea6bf96493a9a5ebfa411565d7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87"
I20260811 16:09:03.186947 10450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7050dea6bf96493a9a5ebfa411565d7f in term 5.
I20260811 16:09:03.192315 11409 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f, 9f927cbce0704eac8a7d3836460dce87; no voters: a17ce76946324f1d9576e7006ab317cc
I20260811 16:09:03.193746 11703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 FOLLOWER]: Leader election won for term 5
I20260811 16:09:03.194231 11703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 LEADER]: Becoming Leader. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Running, Role: LEADER
I20260811 16:09:03.195137 11703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 4.21, Last appended by leader: 21, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.199455 11690 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:09:03.200017 11690 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.202582 11690 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:09:03.205940 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 6 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87" is_pre_election: true
I20260811 16:09:03.206671 10448 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17ce76946324f1d9576e7006ab317cc in term 5.
I20260811 16:09:03.208127 10309 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:09:03.204371 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 6 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f" is_pre_election: true
I20260811 16:09:03.211185 11690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260811 16:09:03.211671 11690 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:09:03.212080 11690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:09:03.215971 11683 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:09:03.216579 11683 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.218844 11683 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:03.219869 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" is_pre_election: true
I20260811 16:09:03.220501 10761 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 7.
I20260811 16:09:03.222162 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:09:03.222060 10877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311" is_pre_election: true
I20260811 16:09:03.223016 10877 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 7.
I20260811 16:09:03.223186 11683 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260811 16:09:03.223627 11683 raft_consensus.cc:482] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:09:03.224944 11683 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 7 FOLLOWER]: Advancing to term 8
I20260811 16:09:03.225910 11690 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.245396 11700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7050dea6bf96493a9a5ebfa411565d7f. Latest consensus state: current_term: 5 leader_uuid: "7050dea6bf96493a9a5ebfa411565d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.246354 11700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:03.255020 11710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:03.256393 11683 raft_consensus.cc:504] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.257313 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 6 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "9f927cbce0704eac8a7d3836460dce87"
I20260811 16:09:03.258049 10450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:09:03.271075 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3f50025c0d14d86a35172b2c6b84b2a"
I20260811 16:09:03.271734 10760 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 7 FOLLOWER]: Advancing to term 8
I20260811 16:09:03.271788 11690 leader_election.cc:288] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 6 election: Requested vote from peers 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181), 7050dea6bf96493a9a5ebfa411565d7f (127.7.228.60:38613)
I20260811 16:09:03.275149 11710 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23]
I20260811 16:09:03.277617 10450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17ce76946324f1d9576e7006ab317cc in term 6.
I20260811 16:09:03.285949 10309 leader_election.cc:302] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc; no voters:
I20260811 16:09:03.286870 11690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 FOLLOWER]: Leader election won for term 6
I20260811 16:09:03.287353 11690 raft_consensus.cc:686] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 LEADER]: Becoming Leader. State: Replica: a17ce76946324f1d9576e7006ab317cc, State: Running, Role: LEADER
I20260811 16:09:03.279963 11683 leader_election.cc:288] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 8 election: Requested vote from peers d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591), 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359)
I20260811 16:09:03.281355 10882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6afe0fe649d34850a77793d9372f92fb" candidate_uuid: "429744fd0c424b86bef5550dccfc6530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52ad56aab4d843c599224c82b3f41311"
I20260811 16:09:03.283908 10760 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 8.
I20260811 16:09:03.288281 11690 consensus_queue.cc:237] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 4.21, Last appended by leader: 21, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.290099 10953 leader_election.cc:302] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429744fd0c424b86bef5550dccfc6530, d3f50025c0d14d86a35172b2c6b84b2a; no voters:
I20260811 16:09:03.291924 11683 raft_consensus.cc:2781] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 8 FOLLOWER]: Leader election won for term 8
I20260811 16:09:03.292443 11683 raft_consensus.cc:686] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 8 LEADER]: Becoming Leader. State: Replica: 429744fd0c424b86bef5550dccfc6530, State: Running, Role: LEADER
I20260811 16:09:03.293113 11710 tablet_loader.cc:96] loaded metadata for tablet 6afe0fe649d34850a77793d9372f92fb (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.288800 10882 raft_consensus.cc:3037] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 7 FOLLOWER]: Advancing to term 8
I20260811 16:09:03.297544 11710 tablet_loader.cc:96] loaded metadata for tablet 84e71aac7b9b46479a8ce87b88b74e45 (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.299156 11710 tablet_loader.cc:96] loaded metadata for tablet d42b5e0273654c95badc4250fc283cbd (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.300709 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a17ce76946324f1d9576e7006ab317cc" candidate_term: 6 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: false dest_uuid: "7050dea6bf96493a9a5ebfa411565d7f"
I20260811 16:09:03.293334 11683 consensus_queue.cc:237] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } }
I20260811 16:09:03.305532 10341 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 LEADER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 5. Current term is 6. Ops: []
I20260811 16:09:03.306138 10882 raft_consensus.cc:2454] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429744fd0c424b86bef5550dccfc6530 in term 8.
I20260811 16:09:03.300657 11710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:03.309213 11700 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260811 16:09:03.309150 11674 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17ce76946324f1d9576e7006ab317cc. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.309916 11700 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [LEADER]: Peer a17ce76946324f1d9576e7006ab317cc log is divergent from this leader: its last log entry 6.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260811 16:09:03.310179 11674 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:03.312286 11713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:03.314138 11710 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:09:03.314517 11710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:03.319516 11713 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23]
I20260811 16:09:03.322594 11703 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 LEADER]: Stepping down as leader of term 5
I20260811 16:09:03.323011 11703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7050dea6bf96493a9a5ebfa411565d7f, State: Running, Role: LEADER
I20260811 16:09:03.323817 11703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 5.22, Last appended by leader: 22, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:03.325335 11703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 5 FOLLOWER]: Advancing to term 6
I20260811 16:09:03.326798 11713 tablet_loader.cc:96] loaded metadata for tablet 6afe0fe649d34850a77793d9372f92fb (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.328387 11713 tablet_loader.cc:96] loaded metadata for tablet 84e71aac7b9b46479a8ce87b88b74e45 (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.330329 11713 tablet_loader.cc:96] loaded metadata for tablet d42b5e0273654c95badc4250fc283cbd (table NewTableName [id=465c3f203d8b47ba84eb9663bfecfe23])
I20260811 16:09:03.331846 11713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:03.333257 11710 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:03.336189 11713 catalog_manager.cc:1259] Loaded cluster ID: 7c5a4baa6b0f40389a5ed6d521344bb8
I20260811 16:09:03.336472 11713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:03.338508 11710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Loaded TSK: 0
I20260811 16:09:03.338985 10450 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 6 FOLLOWER]: Rejecting Update request from peer 7050dea6bf96493a9a5ebfa411565d7f for earlier term 5. Current term is 6. Ops: []
I20260811 16:09:03.340055 11710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:03.341934 11713 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:03.346747 11713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: Loaded TSK: 0
I20260811 16:09:03.348037 11713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:03.379659 11449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Refusing update from remote peer a17ce76946324f1d9576e7006ab317cc: Log matching property violated. Preceding OpId in replica: term: 5 index: 22. Preceding OpId from leader: term: 6 index: 22. (term mismatch)
I20260811 16:09:03.379998 11449 pending_rounds.cc:76] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Aborting all ops after (but not including) 21
I20260811 16:09:03.380187 11449 pending_rounds.cc:97] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f: Aborting uncommitted NO_OP operation due to leader change: 5.22
I20260811 16:09:03.380470 11449 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260811 16:09:03.382071 11690 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260811 16:09:03.382509 10760 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 8 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 16:09:03.384325 11683 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 16:09:03.385824 11714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17ce76946324f1d9576e7006ab317cc. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.386907 11714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:09:03.387957 10804 heartbeater.cc:503] Master 127.7.228.62:43139 was elected leader, sending a full tablet report...
I20260811 16:09:03.389366 10450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 6 FOLLOWER]: Refusing update from remote peer a17ce76946324f1d9576e7006ab317cc: Log matching property violated. Preceding OpId in replica: term: 4 index: 21. Preceding OpId from leader: term: 6 index: 22. (index mismatch)
I20260811 16:09:03.391798 11674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260811 16:09:03.395980 10882 raft_consensus.cc:1261] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 8 FOLLOWER]: Refusing update from remote peer 429744fd0c424b86bef5550dccfc6530: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 16:09:03.401008 11709 consensus_queue.cc:1035] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260811 16:09:03.406541 11690 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.408272 11690 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:03.409224 10930 heartbeater.cc:503] Master 127.7.228.62:43139 was elected leader, sending a full tablet report...
I20260811 16:09:03.426318 10329 catalog_manager.cc:5495] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a reported cstate change: term changed from 6 to 8, leader changed from 52ad56aab4d843c599224c82b3f41311 (127.7.228.2) to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 8 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } }
I20260811 16:09:03.430397 10329 catalog_manager.cc:5495] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a reported cstate change: term changed from 3 to 4, leader changed from 52ad56aab4d843c599224c82b3f41311 (127.7.228.2) to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 4 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } }
I20260811 16:09:03.431988 10329 catalog_manager.cc:5495] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a reported cstate change: term changed from 4 to 5, leader changed from 52ad56aab4d843c599224c82b3f41311 (127.7.228.2) to 429744fd0c424b86bef5550dccfc6530 (127.7.228.3). New cstate: current_term: 5 leader_uuid: "429744fd0c424b86bef5550dccfc6530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f50025c0d14d86a35172b2c6b84b2a" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 34591 } } peers { permanent_uuid: "52ad56aab4d843c599224c82b3f41311" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 40359 } } peers { permanent_uuid: "429744fd0c424b86bef5550dccfc6530" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 37155 } } }
I20260811 16:09:03.437867 11668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17ce76946324f1d9576e7006ab317cc. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.438690 11668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:09:03.447069 11672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.448058 11672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:09:03.452186 11714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.453071 11714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 16:09:03.470534 11690 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "a17ce76946324f1d9576e7006ab317cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } } }
I20260811 16:09:03.472395 11690 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:03.503793 11070 heartbeater.cc:503] Master 127.7.228.62:43139 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:783: Failure
Expected equality of these values:
live_row_count
Which is: 135
table_info->GetMetrics()->live_row_count->value()
Which is: 270
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_util.cc:395: Failure
Failed
Timed out waiting for assertion to pass.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:767: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:785: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 782, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:895: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260811 16:09:11.021072 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
W20260811 16:09:11.044528 10953 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260811 16:09:11.049374 10953 consensus_peers.cc:487] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 -> Peer d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591): Couldn't send request to peer d3f50025c0d14d86a35172b2c6b84b2a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:11.062429 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:11.066119 8080 tablet_replica.cc:330] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a: stopping tablet replica
I20260811 16:09:11.067014 8080 raft_consensus.cc:2229] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.067950 8080 raft_consensus.cc:2258] T d42b5e0273654c95badc4250fc283cbd P d3f50025c0d14d86a35172b2c6b84b2a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.070947 8080 tablet_replica.cc:330] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a: stopping tablet replica
I20260811 16:09:11.071451 8080 raft_consensus.cc:2229] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.071959 8080 raft_consensus.cc:2258] T 84e71aac7b9b46479a8ce87b88b74e45 P d3f50025c0d14d86a35172b2c6b84b2a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.074085 8080 tablet_replica.cc:330] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a: stopping tablet replica
I20260811 16:09:11.074548 8080 raft_consensus.cc:2229] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 8 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.074968 8080 raft_consensus.cc:2258] T 6afe0fe649d34850a77793d9372f92fb P d3f50025c0d14d86a35172b2c6b84b2a [term 8 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.120587 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
W20260811 16:09:11.134435 10953 consensus_peers.cc:487] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 -> Peer d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591): Couldn't send request to peer d3f50025c0d14d86a35172b2c6b84b2a. Status: Network error: Client connection negotiation failed: client connection to 127.7.228.1:34591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 16:09:11.134994 10953 consensus_peers.cc:487] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 -> Peer d3f50025c0d14d86a35172b2c6b84b2a (127.7.228.1:34591): Couldn't send request to peer d3f50025c0d14d86a35172b2c6b84b2a. Status: Network error: Client connection negotiation failed: client connection to 127.7.228.1:34591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:11.141929 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
W20260811 16:09:11.167677 10953 consensus_peers.cc:487] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 -> Peer 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359): Couldn't send request to peer 52ad56aab4d843c599224c82b3f41311. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260811 16:09:11.174729 10953 consensus_peers.cc:487] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 -> Peer 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359): Couldn't send request to peer 52ad56aab4d843c599224c82b3f41311. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:11.178170 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:11.178879 8080 tablet_replica.cc:330] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311: stopping tablet replica
I20260811 16:09:11.179518 8080 raft_consensus.cc:2229] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.180050 8080 raft_consensus.cc:2258] T 6afe0fe649d34850a77793d9372f92fb P 52ad56aab4d843c599224c82b3f41311 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.182756 8080 tablet_replica.cc:330] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311: stopping tablet replica
I20260811 16:09:11.183285 8080 raft_consensus.cc:2229] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.183805 8080 raft_consensus.cc:2258] T d42b5e0273654c95badc4250fc283cbd P 52ad56aab4d843c599224c82b3f41311 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.185959 8080 tablet_replica.cc:330] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311: stopping tablet replica
I20260811 16:09:11.186403 8080 raft_consensus.cc:2229] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.186829 8080 raft_consensus.cc:2258] T 84e71aac7b9b46479a8ce87b88b74e45 P 52ad56aab4d843c599224c82b3f41311 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260811 16:09:11.234889 10953 consensus_peers.cc:487] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 -> Peer 52ad56aab4d843c599224c82b3f41311 (127.7.228.2:40359): Couldn't send request to peer 52ad56aab4d843c599224c82b3f41311. Status: Network error: Client connection negotiation failed: client connection to 127.7.228.2:40359: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:11.236598 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:09:11.255165 8080 tablet_server.cc:178] TabletServer@127.7.228.3:0 shutting down...
I20260811 16:09:11.309330 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:11.310145 8080 tablet_replica.cc:330] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530: stopping tablet replica
I20260811 16:09:11.310837 8080 raft_consensus.cc:2229] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 8 LEADER]: Raft consensus shutting down.
I20260811 16:09:11.311807 8080 raft_consensus.cc:2258] T 6afe0fe649d34850a77793d9372f92fb P 429744fd0c424b86bef5550dccfc6530 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.314535 8080 tablet_replica.cc:330] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530: stopping tablet replica
I20260811 16:09:11.315207 8080 raft_consensus.cc:2229] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 LEADER]: Raft consensus shutting down.
I20260811 16:09:11.315954 8080 raft_consensus.cc:2258] T 84e71aac7b9b46479a8ce87b88b74e45 P 429744fd0c424b86bef5550dccfc6530 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.318518 8080 tablet_replica.cc:330] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530: stopping tablet replica
I20260811 16:09:11.318975 8080 raft_consensus.cc:2229] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 5 LEADER]: Raft consensus shutting down.
I20260811 16:09:11.319720 8080 raft_consensus.cc:2258] T d42b5e0273654c95badc4250fc283cbd P 429744fd0c424b86bef5550dccfc6530 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.343254 8080 tablet_server.cc:194] TabletServer@127.7.228.3:0 shutdown complete.
I20260811 16:09:11.359263 8080 master.cc:537] Master@127.7.228.62:43139 shutting down...
I20260811 16:09:11.389376 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 LEADER]: Raft consensus shutting down.
I20260811 16:09:11.390396 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc [term 6 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.390803 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P a17ce76946324f1d9576e7006ab317cc: stopping tablet replica
I20260811 16:09:11.442180 8080 master.cc:552] Master@127.7.228.62:43139 shutdown complete.
I20260811 16:09:11.464633 8080 master.cc:537] Master@127.7.228.61:34181 shutting down...
I20260811 16:09:11.492493 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.493294 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.493701 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f927cbce0704eac8a7d3836460dce87: stopping tablet replica
I20260811 16:09:11.518781 11758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a17ce76946324f1d9576e7006ab317cc)
I20260811 16:09:11.519332 11758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17ce76946324f1d9576e7006ab317cc" member_type: VOTER last_known_addr { host: "127.7.228.62" port: 43139 } } peers { permanent_uuid: "9f927cbce0704eac8a7d3836460dce87" member_type: VOTER last_known_addr { host: "127.7.228.61" port: 34181 } } peers { permanent_uuid: "7050dea6bf96493a9a5ebfa411565d7f" member_type: VOTER last_known_addr { host: "127.7.228.60" port: 38613 } }
I20260811 16:09:11.521732 11758 leader_election.cc:288] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139), 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181)
W20260811 16:09:11.528326 11409 leader_election.cc:334] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a17ce76946324f1d9576e7006ab317cc (127.7.228.62:43139): Network error: Client connection negotiation failed: client connection to 127.7.228.62:43139: connect: Connection refused (error 111)
W20260811 16:09:11.529125 11409 leader_election.cc:334] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9f927cbce0704eac8a7d3836460dce87 (127.7.228.61:34181): Network error: Client connection negotiation failed: client connection to 127.7.228.61:34181: connect: Connection refused (error 111)
I20260811 16:09:11.529469 11409 leader_election.cc:302] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7050dea6bf96493a9a5ebfa411565d7f; no voters: 9f927cbce0704eac8a7d3836460dce87, a17ce76946324f1d9576e7006ab317cc
I20260811 16:09:11.530258 11758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 16:09:11.553624 8080 master.cc:552] Master@127.7.228.61:34181 shutdown complete.
I20260811 16:09:11.575158 8080 master.cc:537] Master@127.7.228.60:38613 shutting down...
I20260811 16:09:11.601265 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:11.601892 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7050dea6bf96493a9a5ebfa411565d7f [term 6 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:11.602265 8080 tablet_replica.cc:330] stopping tablet replica
I20260811 16:09:11.623833 8080 master.cc:552] Master@127.7.228.60:38613 shutdown complete.
I20260811 16:09:11.651194 8080 test_util.cc:175] -----------------------------------------------
I20260811 16:09:11.651410 8080 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786464498564520-8080-0
[ FAILED ] TsTabletManagerITest.TestTableStats (38763 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260811 16:09:11.658218 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:40413
I20260811 16:09:11.659338 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:11.665485 11772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:11.665702 11774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:11.668718 11776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:11.668833 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:11.670156 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:11.670370 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:11.670517 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464551670502 us; error 0 us; skew 500 ppm
I20260811 16:09:11.671078 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:11.673955 8080 webserver.cc:460] Webserver started at http://127.7.228.62:33701/ using document root <none> and password file <none>
I20260811 16:09:11.674567 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:11.674772 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:11.675061 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:11.676393 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "b942cd1b78624e35a0ea2dcd1314f712"
format_stamp: "Formatted at 2026-08-11 16:09:11 on dist-test-slave-ncp9"
I20260811 16:09:11.681644 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 16:09:11.685460 11783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:11.686401 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 16:09:11.686743 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "b942cd1b78624e35a0ea2dcd1314f712"
format_stamp: "Formatted at 2026-08-11 16:09:11 on dist-test-slave-ncp9"
I20260811 16:09:11.687039 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:11.709815 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:11.711133 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:11.754910 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:40413
I20260811 16:09:11.755069 11867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:40413 every 8 connection(s)
I20260811 16:09:11.759095 11868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:11.770327 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: Bootstrap starting.
I20260811 16:09:11.774955 11868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:11.779556 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: No bootstrap required, opened a new log
I20260811 16:09:11.781797 11868 raft_consensus.cc:348] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER }
I20260811 16:09:11.782238 11868 raft_consensus.cc:374] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:11.782478 11868 raft_consensus.cc:729] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b942cd1b78624e35a0ea2dcd1314f712, State: Initialized, Role: FOLLOWER
I20260811 16:09:11.783084 11868 consensus_queue.cc:260] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER }
I20260811 16:09:11.783551 11868 raft_consensus.cc:388] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:11.783782 11868 raft_consensus.cc:482] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:11.784050 11868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:11.788781 11868 raft_consensus.cc:504] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER }
I20260811 16:09:11.789405 11868 leader_election.cc:302] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b942cd1b78624e35a0ea2dcd1314f712; no voters:
I20260811 16:09:11.790647 11868 leader_election.cc:288] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:11.791054 11875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:11.793367 11875 raft_consensus.cc:686] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 1 LEADER]: Becoming Leader. State: Replica: b942cd1b78624e35a0ea2dcd1314f712, State: Running, Role: LEADER
I20260811 16:09:11.793913 11868 sys_catalog.cc:564] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:11.794088 11875 consensus_queue.cc:237] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER }
I20260811 16:09:11.798454 11877 sys_catalog.cc:455] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b942cd1b78624e35a0ea2dcd1314f712. Latest consensus state: current_term: 1 leader_uuid: "b942cd1b78624e35a0ea2dcd1314f712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER } }
I20260811 16:09:11.798859 11876 sys_catalog.cc:455] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b942cd1b78624e35a0ea2dcd1314f712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b942cd1b78624e35a0ea2dcd1314f712" member_type: VOTER } }
I20260811 16:09:11.799415 11876 sys_catalog.cc:458] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:11.799114 11877 sys_catalog.cc:458] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:11.802628 11886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:11.808450 11886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:11.810140 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:11.817857 11886 catalog_manager.cc:1347] Generated new cluster ID: 78d5510a640a44d7b46fc07cc2bef7b8
I20260811 16:09:11.818125 11886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:11.872016 11886 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:11.873848 11886 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:11.890844 11886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: Generated new TSK 0
I20260811 16:09:11.891546 11886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:11.941519 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:11.948398 11904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:11.949690 11907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:11.951473 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:11.951673 11909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:11.953125 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:11.953341 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:11.953459 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464551953449 us; error 0 us; skew 500 ppm
I20260811 16:09:11.953969 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:11.956444 8080 webserver.cc:460] Webserver started at http://127.7.228.1:36691/ using document root <none> and password file <none>
I20260811 16:09:11.956944 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:11.957124 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:11.957355 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:11.958739 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "db93d635a15042778ec5473d1c762cb1"
format_stamp: "Formatted at 2026-08-11 16:09:11 on dist-test-slave-ncp9"
I20260811 16:09:11.963789 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:09:11.967641 11920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:11.968539 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 16:09:11.968859 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "db93d635a15042778ec5473d1c762cb1"
format_stamp: "Formatted at 2026-08-11 16:09:11 on dist-test-slave-ncp9"
I20260811 16:09:11.969223 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:11.996304 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:11.997750 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:11.999347 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:12.001747 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:12.001955 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.002172 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:12.002353 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.044703 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:46503
I20260811 16:09:12.044806 12016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:46503 every 8 connection(s)
I20260811 16:09:12.049823 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:12.057082 12023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:12.058985 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:12.060534 12028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:12.061682 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:12.062800 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:12.063067 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:12.063279 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464552063250 us; error 0 us; skew 500 ppm
I20260811 16:09:12.063972 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:12.065752 12019 heartbeater.cc:347] Connected to a master server at 127.7.228.62:40413
I20260811 16:09:12.066205 12019 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:12.066834 8080 webserver.cc:460] Webserver started at http://127.7.228.2:39219/ using document root <none> and password file <none>
I20260811 16:09:12.067142 12019 heartbeater.cc:511] Master 127.7.228.62:40413 requested a full tablet report, sending...
I20260811 16:09:12.067462 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:12.067737 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:12.068042 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:12.069576 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "6295bfcce2a946f0a05feb2759052999"
format_stamp: "Formatted at 2026-08-11 16:09:12 on dist-test-slave-ncp9"
I20260811 16:09:12.069587 11807 ts_manager.cc:194] Registered new tserver with Master: db93d635a15042778ec5473d1c762cb1 (127.7.228.1:46503)
I20260811 16:09:12.072540 11807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39846
I20260811 16:09:12.075858 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 16:09:12.079829 12035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.080646 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 16:09:12.080968 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "6295bfcce2a946f0a05feb2759052999"
format_stamp: "Formatted at 2026-08-11 16:09:12 on dist-test-slave-ncp9"
I20260811 16:09:12.081267 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:12.096603 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:12.097834 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:12.099275 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:12.101527 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:12.101722 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.101948 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:12.102087 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.144582 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:39377
I20260811 16:09:12.144760 12126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:39377 every 8 connection(s)
I20260811 16:09:12.149421 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:12.157562 12132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:12.159248 12134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:12.163033 12137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:12.163820 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:12.163830 12127 heartbeater.cc:347] Connected to a master server at 127.7.228.62:40413
I20260811 16:09:12.164469 12127 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:12.165081 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:12.165360 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:12.165385 12127 heartbeater.cc:511] Master 127.7.228.62:40413 requested a full tablet report, sending...
I20260811 16:09:12.165732 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464552165711 us; error 0 us; skew 500 ppm
I20260811 16:09:12.166505 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:12.167989 11807 ts_manager.cc:194] Registered new tserver with Master: 6295bfcce2a946f0a05feb2759052999 (127.7.228.2:39377)
I20260811 16:09:12.169597 8080 webserver.cc:460] Webserver started at http://127.7.228.3:34231/ using document root <none> and password file <none>
I20260811 16:09:12.170300 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:12.170419 11807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39852
I20260811 16:09:12.170563 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:12.170971 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:12.172425 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root/instance:
uuid: "235306341bfa4b9f9b460d7128ecebd3"
format_stamp: "Formatted at 2026-08-11 16:09:12 on dist-test-slave-ncp9"
I20260811 16:09:12.177624 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 16:09:12.181341 12144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.182212 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 16:09:12.182542 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root
uuid: "235306341bfa4b9f9b460d7128ecebd3"
format_stamp: "Formatted at 2026-08-11 16:09:12 on dist-test-slave-ncp9"
I20260811 16:09:12.182925 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:12.212360 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:12.213603 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:12.215049 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:12.217443 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:12.217646 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.217896 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:12.218039 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:12.258641 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.3:35977
I20260811 16:09:12.258740 12235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.3:35977 every 8 connection(s)
I20260811 16:09:12.276306 12237 heartbeater.cc:347] Connected to a master server at 127.7.228.62:40413
I20260811 16:09:12.276721 12237 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:12.277431 12237 heartbeater.cc:511] Master 127.7.228.62:40413 requested a full tablet report, sending...
I20260811 16:09:12.279534 11807 ts_manager.cc:194] Registered new tserver with Master: 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977)
I20260811 16:09:12.279806 8080 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017872831s
I20260811 16:09:12.281353 11807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39856
I20260811 16:09:12.321049 11807 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260811 16:09:12.322695 11807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 16:09:12.365942 11958 tablet_service.cc:1467] Processing CreateTablet for tablet d4a512414e304ba79bde6ff3be493367 (DEFAULT_TABLE table=test-table [id=bc773b02cab841d599a58c6210498cf7]), partition=
I20260811 16:09:12.367237 11958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a512414e304ba79bde6ff3be493367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:12.379274 12186 tablet_service.cc:1467] Processing CreateTablet for tablet d4a512414e304ba79bde6ff3be493367 (DEFAULT_TABLE table=test-table [id=bc773b02cab841d599a58c6210498cf7]), partition=
I20260811 16:09:12.380563 12186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a512414e304ba79bde6ff3be493367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:12.382146 12079 tablet_service.cc:1467] Processing CreateTablet for tablet d4a512414e304ba79bde6ff3be493367 (DEFAULT_TABLE table=test-table [id=bc773b02cab841d599a58c6210498cf7]), partition=
I20260811 16:09:12.383386 12079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a512414e304ba79bde6ff3be493367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:12.386139 12261 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Bootstrap starting.
I20260811 16:09:12.391278 12261 tablet_bootstrap.cc:654] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:12.397022 12261 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: No bootstrap required, opened a new log
I20260811 16:09:12.397545 12261 ts_tablet_manager.cc:1397] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260811 16:09:12.399432 12264 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Bootstrap starting.
I20260811 16:09:12.400977 12261 raft_consensus.cc:348] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.401875 12261 raft_consensus.cc:374] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:12.402161 12266 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Bootstrap starting.
I20260811 16:09:12.402192 12261 raft_consensus.cc:729] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db93d635a15042778ec5473d1c762cb1, State: Initialized, Role: FOLLOWER
I20260811 16:09:12.403292 12261 consensus_queue.cc:260] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.405942 12019 heartbeater.cc:503] Master 127.7.228.62:40413 was elected leader, sending a full tablet report...
I20260811 16:09:12.406075 12261 ts_tablet_manager.cc:1428] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
W20260811 16:09:12.406472 12261 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260811 16:09:12.409017 12266 tablet_bootstrap.cc:654] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:12.410043 12264 tablet_bootstrap.cc:654] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:12.414709 12266 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: No bootstrap required, opened a new log
I20260811 16:09:12.415292 12266 ts_tablet_manager.cc:1397] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260811 16:09:12.418071 12266 raft_consensus.cc:348] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.418889 12266 raft_consensus.cc:374] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:12.419178 12266 raft_consensus.cc:729] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235306341bfa4b9f9b460d7128ecebd3, State: Initialized, Role: FOLLOWER
I20260811 16:09:12.419528 12264 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: No bootstrap required, opened a new log
I20260811 16:09:12.419996 12264 ts_tablet_manager.cc:1397] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.007s
I20260811 16:09:12.419852 12266 consensus_queue.cc:260] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.422360 12237 heartbeater.cc:503] Master 127.7.228.62:40413 was elected leader, sending a full tablet report...
I20260811 16:09:12.422803 12266 ts_tablet_manager.cc:1428] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260811 16:09:12.423123 12266 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260811 16:09:12.422935 12264 raft_consensus.cc:348] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.423672 12264 raft_consensus.cc:374] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:12.423985 12264 raft_consensus.cc:729] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6295bfcce2a946f0a05feb2759052999, State: Initialized, Role: FOLLOWER
I20260811 16:09:12.424575 12264 consensus_queue.cc:260] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.427093 12127 heartbeater.cc:503] Master 127.7.228.62:40413 was elected leader, sending a full tablet report...
I20260811 16:09:12.427644 12264 ts_tablet_manager.cc:1428] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260811 16:09:12.427924 12264 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
W20260811 16:09:12.512828 12241 tablet.cc:2369] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 16:09:12.601553 12277 raft_consensus.cc:482] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:09:12.602118 12277 raft_consensus.cc:504] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.604254 12277 leader_election.cc:288] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db93d635a15042778ec5473d1c762cb1 (127.7.228.1:46503), 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977)
I20260811 16:09:12.619987 11977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a512414e304ba79bde6ff3be493367" candidate_uuid: "6295bfcce2a946f0a05feb2759052999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db93d635a15042778ec5473d1c762cb1" is_pre_election: true
I20260811 16:09:12.619982 12202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a512414e304ba79bde6ff3be493367" candidate_uuid: "6295bfcce2a946f0a05feb2759052999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235306341bfa4b9f9b460d7128ecebd3" is_pre_election: true
I20260811 16:09:12.620960 11977 raft_consensus.cc:2454] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6295bfcce2a946f0a05feb2759052999 in term 0.
I20260811 16:09:12.621027 12202 raft_consensus.cc:2454] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6295bfcce2a946f0a05feb2759052999 in term 0.
I20260811 16:09:12.622241 12043 leader_election.cc:302] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235306341bfa4b9f9b460d7128ecebd3, 6295bfcce2a946f0a05feb2759052999; no voters:
I20260811 16:09:12.622994 12277 raft_consensus.cc:2781] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:09:12.623282 12277 raft_consensus.cc:482] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:09:12.623582 12277 raft_consensus.cc:3037] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:12.629208 12277 raft_consensus.cc:504] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.631258 12277 leader_election.cc:288] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [CANDIDATE]: Term 1 election: Requested vote from peers db93d635a15042778ec5473d1c762cb1 (127.7.228.1:46503), 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977)
I20260811 16:09:12.632136 11977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a512414e304ba79bde6ff3be493367" candidate_uuid: "6295bfcce2a946f0a05feb2759052999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db93d635a15042778ec5473d1c762cb1"
I20260811 16:09:12.632377 12202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a512414e304ba79bde6ff3be493367" candidate_uuid: "6295bfcce2a946f0a05feb2759052999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235306341bfa4b9f9b460d7128ecebd3"
I20260811 16:09:12.632836 11977 raft_consensus.cc:3037] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:12.632930 12202 raft_consensus.cc:3037] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:12.640336 12202 raft_consensus.cc:2454] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6295bfcce2a946f0a05feb2759052999 in term 1.
I20260811 16:09:12.640336 11977 raft_consensus.cc:2454] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6295bfcce2a946f0a05feb2759052999 in term 1.
I20260811 16:09:12.642177 12043 leader_election.cc:302] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235306341bfa4b9f9b460d7128ecebd3, 6295bfcce2a946f0a05feb2759052999; no voters:
I20260811 16:09:12.643297 12277 raft_consensus.cc:2781] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:12.644522 12277 raft_consensus.cc:686] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 LEADER]: Becoming Leader. State: Replica: 6295bfcce2a946f0a05feb2759052999, State: Running, Role: LEADER
I20260811 16:09:12.645358 12277 consensus_queue.cc:237] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.654614 11807 catalog_manager.cc:5495] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6295bfcce2a946f0a05feb2759052999 (127.7.228.2). New cstate: current_term: 1 leader_uuid: "6295bfcce2a946f0a05feb2759052999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260811 16:09:12.732416 12186 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:54316
I20260811 16:09:12.734477 12282 tablet_replica.cc:330] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: stopping tablet replica
I20260811 16:09:12.735142 12282 raft_consensus.cc:2229] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:12.735587 12282 raft_consensus.cc:2258] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:12.737727 12282 ts_tablet_manager.cc:1905] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 16:09:12.748523 12282 ts_tablet_manager.cc:1918] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260811 16:09:12.748961 12282 log.cc:1196] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root/wals/d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.753621 12284 ts_tablet_manager.cc:927] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Initiating tablet copy from peer 6295bfcce2a946f0a05feb2759052999 (127.7.228.2:39377)
I20260811 16:09:12.755841 12284 tablet_copy_client.cc:316] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet copy: Beginning tablet copy session from remote peer at address 127.7.228.2:39377
I20260811 16:09:12.767009 12108 tablet_copy_service.cc:140] P 6295bfcce2a946f0a05feb2759052999: Received BeginTabletCopySession request for tablet d4a512414e304ba79bde6ff3be493367 from peer 235306341bfa4b9f9b460d7128ecebd3 ({username='slave'} at 127.0.0.1:44906)
I20260811 16:09:12.767611 12108 tablet_copy_service.cc:161] P 6295bfcce2a946f0a05feb2759052999: Beginning new tablet copy session on tablet d4a512414e304ba79bde6ff3be493367 from peer 235306341bfa4b9f9b460d7128ecebd3 at {username='slave'} at 127.0.0.1:44906: session id = 235306341bfa4b9f9b460d7128ecebd3-d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.773445 12108 tablet_copy_source_session.cc:205] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 16:09:12.776304 12284 ts_tablet_manager.cc:1905] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260811 16:09:12.787634 12284 ts_tablet_manager.cc:1918] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260811 16:09:12.788247 12284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a512414e304ba79bde6ff3be493367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:12.793741 12284 tablet_copy_client.cc:791] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet copy: Starting download of 0 data blocks...
I20260811 16:09:12.794328 12284 tablet_copy_client.cc:655] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet copy: Starting download of 1 WAL segments...
I20260811 16:09:12.796274 11807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39866:
table { table_id: "bc773b02cab841d599a58c6210498cf7" table_name: "test-table" }
I20260811 16:09:12.797030 11807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39866:
table { table_id: "bc773b02cab841d599a58c6210498cf7" table_name: "test-table" }
I20260811 16:09:12.798568 12284 tablet_copy_client.cc:531] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 16:09:12.805529 12284 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Bootstrap starting.
I20260811 16:09:12.807837 11807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: Sending DeleteTablet for 3 replicas of tablet d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.810027 11958 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 16:09:12 UTC) from {username='slave'} at 127.0.0.1:41516
I20260811 16:09:12.810223 12186 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 16:09:12 UTC) from {username='slave'} at 127.0.0.1:54298
I20260811 16:09:12.811167 12079 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 16:09:12 UTC) from {username='slave'} at 127.0.0.1:44896
W20260811 16:09:12.812417 11790 catalog_manager.cc:4821] TS 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977): delete failed for tablet d4a512414e304ba79bde6ff3be493367 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4a512414e304ba79bde6ff3be493367 already in progress: copying tablet
I20260811 16:09:12.814294 12286 tablet_replica.cc:330] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: stopping tablet replica
I20260811 16:09:12.815117 12286 raft_consensus.cc:2229] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:12.815820 12286 raft_consensus.cc:2258] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:12.816097 12287 tablet_replica.cc:330] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: stopping tablet replica
I20260811 16:09:12.816998 12287 raft_consensus.cc:2229] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:12.817860 12287 pending_rounds.cc:62] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Trying to abort 1 pending ops.
I20260811 16:09:12.818199 12287 pending_rounds.cc:68] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317358807642009600 op_type: NO_OP noop_request { }
I20260811 16:09:12.818778 12287 raft_consensus.cc:2866] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 16:09:12.818857 12286 ts_tablet_manager.cc:1905] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 16:09:12.819233 12287 raft_consensus.cc:2258] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:12.822386 12287 ts_tablet_manager.cc:1905] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 16:09:12.823184 12284 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 16:09:12.824133 12284 tablet_bootstrap.cc:492] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Bootstrap complete.
I20260811 16:09:12.824875 12284 ts_tablet_manager.cc:1397] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20260811 16:09:12.827872 12284 raft_consensus.cc:348] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.828867 12284 raft_consensus.cc:729] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235306341bfa4b9f9b460d7128ecebd3, State: Initialized, Role: FOLLOWER
I20260811 16:09:12.829707 12284 consensus_queue.cc:260] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db93d635a15042778ec5473d1c762cb1" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 46503 } } peers { permanent_uuid: "235306341bfa4b9f9b460d7128ecebd3" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35977 } } peers { permanent_uuid: "6295bfcce2a946f0a05feb2759052999" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 39377 } }
I20260811 16:09:12.831287 12286 ts_tablet_manager.cc:1918] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 16:09:12.831712 12286 log.cc:1196] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-0-root/wals/d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.831884 12284 ts_tablet_manager.cc:1428] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Time spent starting tablet: real 0.007s user 0.001s sys 0.003s
W20260811 16:09:12.832171 12284 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260811 16:09:12.832379 12286 ts_tablet_manager.cc:1939] T d4a512414e304ba79bde6ff3be493367 P db93d635a15042778ec5473d1c762cb1: Deleting consensus metadata
I20260811 16:09:12.835309 11785 catalog_manager.cc:4841] TS db93d635a15042778ec5473d1c762cb1 (127.7.228.1:46503): tablet d4a512414e304ba79bde6ff3be493367 (table test-table [id=bc773b02cab841d599a58c6210498cf7]) successfully deleted
I20260811 16:09:12.837659 12186 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 16:09:12 UTC) from {username='slave'} at 127.0.0.1:54298
I20260811 16:09:12.837909 12287 ts_tablet_manager.cc:1918] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 16:09:12.838335 12287 log.cc:1196] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-1-root/wals/d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.839007 12287 ts_tablet_manager.cc:1939] T d4a512414e304ba79bde6ff3be493367 P 6295bfcce2a946f0a05feb2759052999: Deleting consensus metadata
W20260811 16:09:12.839061 11790 catalog_manager.cc:4821] TS 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977): delete failed for tablet d4a512414e304ba79bde6ff3be493367 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4a512414e304ba79bde6ff3be493367 already in progress: copying tablet
I20260811 16:09:12.840463 11786 catalog_manager.cc:4841] TS 6295bfcce2a946f0a05feb2759052999 (127.7.228.2:39377): tablet d4a512414e304ba79bde6ff3be493367 (table test-table [id=bc773b02cab841d599a58c6210498cf7]) successfully deleted
I20260811 16:09:12.884410 12108 tablet_copy_service.cc:342] P 6295bfcce2a946f0a05feb2759052999: Request end of tablet copy session 235306341bfa4b9f9b460d7128ecebd3-d4a512414e304ba79bde6ff3be493367 received from {username='slave'} at 127.0.0.1:44906
I20260811 16:09:12.884863 12108 tablet_copy_service.cc:433] P 6295bfcce2a946f0a05feb2759052999: ending tablet copy session 235306341bfa4b9f9b460d7128ecebd3-d4a512414e304ba79bde6ff3be493367 on tablet d4a512414e304ba79bde6ff3be493367 with peer 235306341bfa4b9f9b460d7128ecebd3
I20260811 16:09:12.889050 12186 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a512414e304ba79bde6ff3be493367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 16:09:12 UTC) from {username='slave'} at 127.0.0.1:54298
I20260811 16:09:12.889803 12282 tablet_replica.cc:330] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: stopping tablet replica
I20260811 16:09:12.890368 12282 raft_consensus.cc:2229] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:12.890733 12282 pending_rounds.cc:62] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Trying to abort 1 pending ops.
I20260811 16:09:12.890911 12282 pending_rounds.cc:68] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317358807642009600 op_type: NO_OP noop_request { }
I20260811 16:09:12.891211 12282 raft_consensus.cc:2866] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 16:09:12.891523 12282 raft_consensus.cc:2258] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:12.893903 12282 ts_tablet_manager.cc:1905] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 16:09:12.904765 12282 ts_tablet_manager.cc:1918] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 16:09:12.905133 12282 log.cc:1196] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786464498564520-8080-0/minicluster-data/ts-2-root/wals/d4a512414e304ba79bde6ff3be493367
I20260811 16:09:12.905802 12282 ts_tablet_manager.cc:1939] T d4a512414e304ba79bde6ff3be493367 P 235306341bfa4b9f9b460d7128ecebd3: Deleting consensus metadata
I20260811 16:09:12.908301 11790 catalog_manager.cc:4841] TS 235306341bfa4b9f9b460d7128ecebd3 (127.7.228.3:35977): tablet d4a512414e304ba79bde6ff3be493367 (table test-table [id=bc773b02cab841d599a58c6210498cf7]) successfully deleted
I20260811 16:09:12.951896 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:12.971746 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:12.988858 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:12.998805 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:09:13.018802 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:13.036434 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:09:13.046211 8080 tablet_server.cc:178] TabletServer@127.7.228.3:0 shutting down...
I20260811 16:09:13.064500 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:13.081913 8080 tablet_server.cc:194] TabletServer@127.7.228.3:0 shutdown complete.
I20260811 16:09:13.092299 8080 master.cc:537] Master@127.7.228.62:40413 shutting down...
I20260811 16:09:13.109179 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:13.109771 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:13.110105 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P b942cd1b78624e35a0ea2dcd1314f712: stopping tablet replica
I20260811 16:09:13.129747 8080 master.cc:552] Master@127.7.228.62:40413 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1501 ms)
[----------] 4 tests from TsTabletManagerITest (43989 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260811 16:09:13.159489 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:38087
I20260811 16:09:13.160524 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:13.165796 12293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:13.166684 12295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:13.168040 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:13.169102 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:13.169874 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:13.170053 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:13.170192 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464553170180 us; error 0 us; skew 500 ppm
I20260811 16:09:13.170789 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:13.173331 8080 webserver.cc:460] Webserver started at http://127.7.228.62:46533/ using document root <none> and password file <none>
I20260811 16:09:13.173843 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:13.174013 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:13.174280 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:13.175416 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "9867cc97e1144987b21cdddf631577b3"
format_stamp: "Formatted at 2026-08-11 16:09:13 on dist-test-slave-ncp9"
I20260811 16:09:13.180365 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 16:09:13.184549 12306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:13.185395 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:09:13.185710 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "9867cc97e1144987b21cdddf631577b3"
format_stamp: "Formatted at 2026-08-11 16:09:13 on dist-test-slave-ncp9"
I20260811 16:09:13.186003 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:13.213852 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:13.214931 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:13.252367 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:38087
I20260811 16:09:13.252482 12382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:38087 every 8 connection(s)
I20260811 16:09:13.256423 12383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:13.267155 12383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3: Bootstrap starting.
I20260811 16:09:13.271489 12383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:13.275797 12383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3: No bootstrap required, opened a new log
I20260811 16:09:13.277896 12383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER }
I20260811 16:09:13.278306 12383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:13.278542 12383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9867cc97e1144987b21cdddf631577b3, State: Initialized, Role: FOLLOWER
I20260811 16:09:13.279197 12383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER }
I20260811 16:09:13.279677 12383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:13.279887 12383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:13.280134 12383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:13.284855 12383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER }
I20260811 16:09:13.285456 12383 leader_election.cc:302] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9867cc97e1144987b21cdddf631577b3; no voters:
I20260811 16:09:13.286820 12383 leader_election.cc:288] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:13.287089 12386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:13.288502 12386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 1 LEADER]: Becoming Leader. State: Replica: 9867cc97e1144987b21cdddf631577b3, State: Running, Role: LEADER
I20260811 16:09:13.289223 12386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER }
I20260811 16:09:13.290005 12383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:13.292724 12387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9867cc97e1144987b21cdddf631577b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER } }
I20260811 16:09:13.293581 12387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:13.292827 12389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9867cc97e1144987b21cdddf631577b3. Latest consensus state: current_term: 1 leader_uuid: "9867cc97e1144987b21cdddf631577b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9867cc97e1144987b21cdddf631577b3" member_type: VOTER } }
I20260811 16:09:13.294407 12389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:13.297034 12400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:13.301925 12400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:13.305698 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:13.311028 12400 catalog_manager.cc:1347] Generated new cluster ID: 525406a737644258b096f222c405cd6d
I20260811 16:09:13.311362 12400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:13.363265 12400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:13.365109 12400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:13.381737 12400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3: Generated new TSK 0
I20260811 16:09:13.382504 12400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:13.437016 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:13.443616 12419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:13.444648 12420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:13.454833 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:13.456053 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:13.457152 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:13.457394 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:13.457538 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464553457525 us; error 0 us; skew 500 ppm
I20260811 16:09:13.458081 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:13.460546 8080 webserver.cc:460] Webserver started at http://127.7.228.1:34463/ using document root <none> and password file <none>
I20260811 16:09:13.461102 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:13.461303 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:13.461556 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:13.462715 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "19a214d4492542e2a16ba963e291eaf8"
format_stamp: "Formatted at 2026-08-11 16:09:13 on dist-test-slave-ncp9"
I20260811 16:09:13.467499 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:09:13.471068 12429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:13.471871 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:09:13.472186 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "19a214d4492542e2a16ba963e291eaf8"
format_stamp: "Formatted at 2026-08-11 16:09:13 on dist-test-slave-ncp9"
I20260811 16:09:13.472465 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:13.484864 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:13.486070 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:13.491758 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:13.492063 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:13.492342 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:13.492558 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:13.550288 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:38547
I20260811 16:09:13.550413 12542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:38547 every 8 connection(s)
I20260811 16:09:13.554950 12546 heartbeater.cc:347] Connected to a master server at 127.7.228.62:38087
I20260811 16:09:13.555369 12546 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:13.556134 12546 heartbeater.cc:511] Master 127.7.228.62:38087 requested a full tablet report, sending...
I20260811 16:09:13.558099 12330 ts_manager.cc:194] Registered new tserver with Master: 19a214d4492542e2a16ba963e291eaf8 (127.7.228.1:38547)
I20260811 16:09:13.558382 8080 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004902983s
I20260811 16:09:13.560251 12330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49740
I20260811 16:09:13.597023 12477 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 16:09:13.598491 12477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:13.611387 12560 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Bootstrap starting.
I20260811 16:09:13.616014 12560 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:13.620595 12560 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: No bootstrap required, opened a new log
I20260811 16:09:13.621053 12560 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 16:09:13.622949 12560 raft_consensus.cc:348] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.623358 12560 raft_consensus.cc:374] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:13.623545 12560 raft_consensus.cc:729] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a214d4492542e2a16ba963e291eaf8, State: Initialized, Role: FOLLOWER
I20260811 16:09:13.624035 12560 consensus_queue.cc:260] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.624492 12560 raft_consensus.cc:388] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:13.624689 12560 raft_consensus.cc:482] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:13.624928 12560 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:13.630017 12560 raft_consensus.cc:504] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.630599 12560 leader_election.cc:302] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a214d4492542e2a16ba963e291eaf8; no voters:
I20260811 16:09:13.631837 12560 leader_election.cc:288] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:13.632225 12563 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:13.633630 12563 raft_consensus.cc:686] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 LEADER]: Becoming Leader. State: Replica: 19a214d4492542e2a16ba963e291eaf8, State: Running, Role: LEADER
I20260811 16:09:13.633880 12546 heartbeater.cc:503] Master 127.7.228.62:38087 was elected leader, sending a full tablet report...
I20260811 16:09:13.634317 12560 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260811 16:09:13.634408 12563 consensus_queue.cc:237] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.638063 12565 tablet_replica.cc:439] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: TxnStatusTablet state changed. Reason: New leader 19a214d4492542e2a16ba963e291eaf8. Latest consensus state: current_term: 1 leader_uuid: "19a214d4492542e2a16ba963e291eaf8" committed_config { opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } } }
I20260811 16:09:13.638789 12565 tablet_replica.cc:442] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:13.639483 12564 tablet_replica.cc:439] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19a214d4492542e2a16ba963e291eaf8" committed_config { opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } } }
I20260811 16:09:13.640195 12564 tablet_replica.cc:442] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: This TxnStatusTablet replica's current role is: LEADER
W20260811 16:09:13.642217 12330 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:13.642251 12570 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:13.642859 12570 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260811 16:09:13.647413 12330 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:13.679515 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:13.711674 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:13.712482 8080 tablet_replica.cc:330] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: stopping tablet replica
I20260811 16:09:13.713138 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:13.713676 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:13.737916 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:13.750577 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:13.756770 12575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:13.757076 12576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:13.760486 12584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:13.761420 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:13.762776 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:13.763044 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:13.763234 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464553763218 us; error 0 us; skew 500 ppm
I20260811 16:09:13.763972 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:13.767196 8080 webserver.cc:460] Webserver started at http://127.7.228.1:34463/ using document root <none> and password file <none>
I20260811 16:09:13.767844 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:13.768080 8080 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260811 16:09:13.773144 8080 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 16:09:13.777014 12590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:13.777930 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 16:09:13.778319 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "19a214d4492542e2a16ba963e291eaf8"
format_stamp: "Formatted at 2026-08-11 16:09:13 on dist-test-slave-ncp9"
I20260811 16:09:13.778777 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:13.812793 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:13.814260 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:13.821962 12606 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 16:09:13.830327 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 16:09:13.830791 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.002s
I20260811 16:09:13.831141 8080 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 16:09:13.837797 8080 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 16:09:13.838071 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.002s sys 0.004s
I20260811 16:09:13.838418 12606 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Bootstrap starting.
I20260811 16:09:13.872185 12606 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:09:13.873598 12606 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Bootstrap complete.
I20260811 16:09:13.874708 12606 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Time spent bootstrapping tablet: real 0.037s user 0.029s sys 0.004s
I20260811 16:09:13.877375 12606 raft_consensus.cc:348] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.878050 12606 raft_consensus.cc:729] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a214d4492542e2a16ba963e291eaf8, State: Initialized, Role: FOLLOWER
I20260811 16:09:13.878944 12606 consensus_queue.cc:260] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.879912 12606 raft_consensus.cc:388] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:13.880441 12606 raft_consensus.cc:482] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:13.881111 12606 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:09:13.898590 12606 raft_consensus.cc:504] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.899523 12606 leader_election.cc:302] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a214d4492542e2a16ba963e291eaf8; no voters:
I20260811 16:09:13.910944 12606 leader_election.cc:288] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 16:09:13.910261 12619 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:09:13.926936 12619 raft_consensus.cc:686] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 2 LEADER]: Becoming Leader. State: Replica: 19a214d4492542e2a16ba963e291eaf8, State: Running, Role: LEADER
I20260811 16:09:13.928642 12606 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8: Time spent starting tablet: real 0.053s user 0.011s sys 0.023s
I20260811 16:09:13.930003 12619 consensus_queue.cc:237] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } }
I20260811 16:09:13.933610 12621 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "19a214d4492542e2a16ba963e291eaf8" committed_config { opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } } }
I20260811 16:09:13.933564 12624 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 19a214d4492542e2a16ba963e291eaf8. Latest consensus state: current_term: 2 leader_uuid: "19a214d4492542e2a16ba963e291eaf8" committed_config { opid_index: -1 peers { permanent_uuid: "19a214d4492542e2a16ba963e291eaf8" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38547 } } }
I20260811 16:09:13.934341 12621 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:13.934374 12624 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:13.948103 12641 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:13.948684 12641 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 16:09:13.992967 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:38547
I20260811 16:09:13.993166 12717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:38547 every 8 connection(s)
I20260811 16:09:13.999369 8080 tablet_server.cc:178] TabletServer@127.7.228.1:38547 shutting down...
I20260811 16:09:14.002780 12719 heartbeater.cc:347] Connected to a master server at 127.7.228.62:38087
I20260811 16:09:14.006297 12719 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:14.007594 12719 heartbeater.cc:511] Master 127.7.228.62:38087 requested a full tablet report, sending...
I20260811 16:09:14.010483 12330 ts_manager.cc:194] Re-registered known tserver with Master: 19a214d4492542e2a16ba963e291eaf8 (127.7.228.1:38547)
W20260811 16:09:14.010989 12330 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:14.012729 12330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49770
I20260811 16:09:14.018290 12719 heartbeater.cc:503] Master 127.7.228.62:38087 was elected leader, sending a full tablet report...
W20260811 16:09:14.021988 12330 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:14.035171 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:14.036262 8080 tablet_replica.cc:330] stopping tablet replica
I20260811 16:09:14.036854 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 2 LEADER]: Raft consensus shutting down.
I20260811 16:09:14.037410 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 19a214d4492542e2a16ba963e291eaf8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:14.049551 8080 tablet_server.cc:194] TabletServer@127.7.228.1:38547 shutdown complete.
I20260811 16:09:14.061638 8080 master.cc:537] Master@127.7.228.62:38087 shutting down...
I20260811 16:09:14.080459 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:14.081185 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:14.081581 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9867cc97e1144987b21cdddf631577b3: stopping tablet replica
I20260811 16:09:14.101045 8080 master.cc:552] Master@127.7.228.62:38087 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (965 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260811 16:09:14.126080 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:33449
I20260811 16:09:14.127215 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:14.133075 12726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:14.135649 12729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:14.136307 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:14.137060 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:14.137856 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:14.138033 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:14.138200 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464554138186 us; error 0 us; skew 500 ppm
I20260811 16:09:14.138756 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:14.141644 8080 webserver.cc:460] Webserver started at http://127.7.228.62:46687/ using document root <none> and password file <none>
I20260811 16:09:14.142167 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:14.142364 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:14.142639 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:14.143784 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "d2520b159ead444bbc2c680da9afad9f"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.149138 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 16:09:14.152876 12745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.153821 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 16:09:14.154192 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "d2520b159ead444bbc2c680da9afad9f"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.154570 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:14.197570 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:14.198652 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:14.249117 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:33449
I20260811 16:09:14.249233 12828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:33449 every 8 connection(s)
I20260811 16:09:14.253690 12831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:14.269737 12831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f: Bootstrap starting.
I20260811 16:09:14.275818 12831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:14.280781 12831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f: No bootstrap required, opened a new log
I20260811 16:09:14.282872 12831 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER }
I20260811 16:09:14.283331 12831 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:14.283565 12831 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2520b159ead444bbc2c680da9afad9f, State: Initialized, Role: FOLLOWER
I20260811 16:09:14.284132 12831 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER }
I20260811 16:09:14.284638 12831 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:14.284878 12831 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:14.285168 12831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:14.289935 12831 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER }
I20260811 16:09:14.290508 12831 leader_election.cc:302] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2520b159ead444bbc2c680da9afad9f; no voters:
I20260811 16:09:14.291903 12831 leader_election.cc:288] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:14.292255 12834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:14.294107 12834 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 1 LEADER]: Becoming Leader. State: Replica: d2520b159ead444bbc2c680da9afad9f, State: Running, Role: LEADER
I20260811 16:09:14.294941 12834 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER }
I20260811 16:09:14.295430 12831 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:14.299129 12835 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2520b159ead444bbc2c680da9afad9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER } }
I20260811 16:09:14.299338 12840 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2520b159ead444bbc2c680da9afad9f. Latest consensus state: current_term: 1 leader_uuid: "d2520b159ead444bbc2c680da9afad9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2520b159ead444bbc2c680da9afad9f" member_type: VOTER } }
I20260811 16:09:14.299751 12835 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:14.299973 12840 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:14.310374 12847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:14.317663 12847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:14.318770 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:14.328362 12847 catalog_manager.cc:1347] Generated new cluster ID: 993c72dcf6b141a78a1b8b7f841b133e
I20260811 16:09:14.328698 12847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:14.345434 12847 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:14.347173 12847 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:14.360185 12847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f: Generated new TSK 0
I20260811 16:09:14.360996 12847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:14.385941 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:14.393445 12871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:14.394996 12872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:14.398236 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:14.398780 12876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:14.399847 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:14.400075 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:14.400233 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464554400215 us; error 0 us; skew 500 ppm
I20260811 16:09:14.400777 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:14.403935 8080 webserver.cc:460] Webserver started at http://127.7.228.1:42397/ using document root <none> and password file <none>
I20260811 16:09:14.404443 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:14.404635 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:14.404938 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:14.406109 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "c91be5949ff341cd986777111ba1d16f"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.411394 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 16:09:14.416004 12884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.417034 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:09:14.417444 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "c91be5949ff341cd986777111ba1d16f"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.417840 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:14.448995 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:14.450348 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:14.463150 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:14.463642 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.001s
I20260811 16:09:14.464004 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:14.464278 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.540678 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:38913
I20260811 16:09:14.540800 12992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:38913 every 8 connection(s)
I20260811 16:09:14.546653 12993 heartbeater.cc:347] Connected to a master server at 127.7.228.62:33449
I20260811 16:09:14.546684 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 16:09:14.547159 12993 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:14.547878 12993 heartbeater.cc:511] Master 127.7.228.62:33449 requested a full tablet report, sending...
I20260811 16:09:14.549860 12774 ts_manager.cc:194] Registered new tserver with Master: c91be5949ff341cd986777111ba1d16f (127.7.228.1:38913)
I20260811 16:09:14.551806 12774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57676
W20260811 16:09:14.554847 12995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:14.558041 13000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:14.561826 13003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:14.564679 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:14.565877 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:14.566079 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:14.566248 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464554566232 us; error 0 us; skew 500 ppm
I20260811 16:09:14.566941 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:14.570753 8080 webserver.cc:460] Webserver started at http://127.7.228.2:42747/ using document root <none> and password file <none>
I20260811 16:09:14.571254 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:14.571442 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:14.571696 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:14.572795 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "b31ddf2f0ba043d283bf5b5e3dca7ad1"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.577739 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 16:09:14.581382 13008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.582320 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 16:09:14.582654 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "b31ddf2f0ba043d283bf5b5e3dca7ad1"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:14.582993 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:14.596429 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:14.597640 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:14.604518 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:14.604830 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.605290 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:14.605521 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:14.669991 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:44787
I20260811 16:09:14.670117 13130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:44787 every 8 connection(s)
I20260811 16:09:14.675215 13132 heartbeater.cc:347] Connected to a master server at 127.7.228.62:33449
I20260811 16:09:14.675676 13132 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:14.676678 13132 heartbeater.cc:511] Master 127.7.228.62:33449 requested a full tablet report, sending...
I20260811 16:09:14.678920 12774 ts_manager.cc:194] Registered new tserver with Master: b31ddf2f0ba043d283bf5b5e3dca7ad1 (127.7.228.2:44787)
I20260811 16:09:14.679917 8080 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.006420602s
I20260811 16:09:14.680712 12774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57700
I20260811 16:09:14.724520 12932 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 16:09:14.726178 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:14.742735 13146 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Bootstrap starting.
I20260811 16:09:14.747376 13146 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:14.751765 13146 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: No bootstrap required, opened a new log
I20260811 16:09:14.752189 13146 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 16:09:14.754163 13146 raft_consensus.cc:348] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.754637 13146 raft_consensus.cc:374] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:14.754842 13146 raft_consensus.cc:729] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91be5949ff341cd986777111ba1d16f, State: Initialized, Role: FOLLOWER
I20260811 16:09:14.755503 13146 consensus_queue.cc:260] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.756040 13146 raft_consensus.cc:388] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:14.756321 13146 raft_consensus.cc:482] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:14.756601 13146 raft_consensus.cc:3037] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:14.761320 13146 raft_consensus.cc:504] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.761905 13146 leader_election.cc:302] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c91be5949ff341cd986777111ba1d16f; no voters:
I20260811 16:09:14.763307 13146 leader_election.cc:288] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:14.763723 13150 raft_consensus.cc:2781] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:14.765437 12993 heartbeater.cc:503] Master 127.7.228.62:33449 was elected leader, sending a full tablet report...
I20260811 16:09:14.765599 13150 raft_consensus.cc:686] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 LEADER]: Becoming Leader. State: Replica: c91be5949ff341cd986777111ba1d16f, State: Running, Role: LEADER
I20260811 16:09:14.766398 13150 consensus_queue.cc:237] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.767040 13146 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 16:09:14.770522 13152 tablet_replica.cc:439] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: TxnStatusTablet state changed. Reason: New leader c91be5949ff341cd986777111ba1d16f. Latest consensus state: current_term: 1 leader_uuid: "c91be5949ff341cd986777111ba1d16f" committed_config { opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } } }
I20260811 16:09:14.771129 13152 tablet_replica.cc:442] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: This TxnStatusTablet replica's current role is: LEADER
W20260811 16:09:14.773334 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:14.775736 13151 tablet_replica.cc:439] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c91be5949ff341cd986777111ba1d16f" committed_config { opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } } }
I20260811 16:09:14.776352 13151 tablet_replica.cc:442] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:14.779676 13156 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:14.780262 13156 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260811 16:09:14.783072 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:14.841401 13161 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Initiating tablet copy from peer c91be5949ff341cd986777111ba1d16f (127.7.228.1:38913)
I20260811 16:09:14.843197 13161 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.228.1:38913
I20260811 16:09:14.855381 12969 tablet_copy_service.cc:140] P c91be5949ff341cd986777111ba1d16f: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer b31ddf2f0ba043d283bf5b5e3dca7ad1 ({username='slave'} at 127.0.0.1:47698)
I20260811 16:09:14.856101 12969 tablet_copy_service.cc:161] P c91be5949ff341cd986777111ba1d16f: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer b31ddf2f0ba043d283bf5b5e3dca7ad1 at {username='slave'} at 127.0.0.1:47698: session id = b31ddf2f0ba043d283bf5b5e3dca7ad1-11111111111111111111111111111111
I20260811 16:09:14.864362 12969 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 16:09:14.867506 13161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:14.878810 13161 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: tablet copy: Starting download of 0 data blocks...
I20260811 16:09:14.879465 13161 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: tablet copy: Starting download of 1 WAL segments...
I20260811 16:09:14.883111 13161 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 16:09:14.889866 13161 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Bootstrap starting.
I20260811 16:09:14.912838 13161 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:09:14.913681 13161 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Bootstrap complete.
I20260811 16:09:14.914291 13161 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20260811 16:09:14.916566 13161 raft_consensus.cc:348] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.917109 13161 raft_consensus.cc:729] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: b31ddf2f0ba043d283bf5b5e3dca7ad1, State: Initialized, Role: NON_PARTICIPANT
I20260811 16:09:14.917594 13161 consensus_queue.cc:260] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:14.919384 13164 tablet_replica.cc:439] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } } }
I20260811 16:09:14.919653 13132 heartbeater.cc:503] Master 127.7.228.62:33449 was elected leader, sending a full tablet report...
I20260811 16:09:14.920087 13164 tablet_replica.cc:442] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260811 16:09:14.920250 13161 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 16:09:14.922178 12969 tablet_copy_service.cc:342] P c91be5949ff341cd986777111ba1d16f: Request end of tablet copy session b31ddf2f0ba043d283bf5b5e3dca7ad1-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:47698
I20260811 16:09:14.922530 12969 tablet_copy_service.cc:433] P c91be5949ff341cd986777111ba1d16f: ending tablet copy session b31ddf2f0ba043d283bf5b5e3dca7ad1-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer b31ddf2f0ba043d283bf5b5e3dca7ad1
W20260811 16:09:14.923098 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260811 16:09:14.927886 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:14.944373 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:14.978647 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:14.979547 8080 tablet_replica.cc:330] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: stopping tablet replica
I20260811 16:09:14.980185 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:14.980645 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:15.004513 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:15.021474 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:15.029106 13167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.029233 13168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.035508 13171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:15.035625 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:15.037317 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:15.037614 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:15.037861 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464555037843 us; error 0 us; skew 500 ppm
I20260811 16:09:15.038578 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:15.042096 8080 webserver.cc:460] Webserver started at http://127.7.228.1:42397/ using document root <none> and password file <none>
I20260811 16:09:15.042814 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:15.043108 8080 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260811 16:09:15.048321 8080 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:09:15.052273 13178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:15.053222 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 16:09:15.053560 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "c91be5949ff341cd986777111ba1d16f"
format_stamp: "Formatted at 2026-08-11 16:09:14 on dist-test-slave-ncp9"
I20260811 16:09:15.053857 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:15.130415 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:15.131471 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:15.138159 13192 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 16:09:15.145682 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 16:09:15.146051 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.002s sys 0.000s
I20260811 16:09:15.146390 8080 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 16:09:15.152961 8080 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 16:09:15.153244 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.005s sys 0.003s
I20260811 16:09:15.154035 13192 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Bootstrap starting.
I20260811 16:09:15.193887 13192 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:09:15.194957 13192 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Bootstrap complete.
I20260811 16:09:15.195709 13192 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Time spent bootstrapping tablet: real 0.042s user 0.029s sys 0.000s
I20260811 16:09:15.198688 13192 raft_consensus.cc:348] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:15.199328 13192 raft_consensus.cc:729] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91be5949ff341cd986777111ba1d16f, State: Initialized, Role: FOLLOWER
I20260811 16:09:15.200109 13192 consensus_queue.cc:260] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:15.200783 13192 raft_consensus.cc:388] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:15.201140 13192 raft_consensus.cc:482] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:15.201496 13192 raft_consensus.cc:3037] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 1 FOLLOWER]: Advancing to term 2
I20260811 16:09:15.209921 13192 raft_consensus.cc:504] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:15.210714 13192 leader_election.cc:302] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c91be5949ff341cd986777111ba1d16f; no voters:
I20260811 16:09:15.212473 13192 leader_election.cc:288] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 16:09:15.212882 13223 raft_consensus.cc:2781] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 16:09:15.216169 13223 raft_consensus.cc:686] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 2 LEADER]: Becoming Leader. State: Replica: c91be5949ff341cd986777111ba1d16f, State: Running, Role: LEADER
I20260811 16:09:15.217130 13223 consensus_queue.cc:237] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } }
I20260811 16:09:15.222941 13192 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f: Time spent starting tablet: real 0.027s user 0.020s sys 0.004s
I20260811 16:09:15.222961 13229 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c91be5949ff341cd986777111ba1d16f" committed_config { opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } } }
I20260811 16:09:15.223891 13229 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:15.225989 13227 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader c91be5949ff341cd986777111ba1d16f. Latest consensus state: current_term: 2 leader_uuid: "c91be5949ff341cd986777111ba1d16f" committed_config { opid_index: -1 peers { permanent_uuid: "c91be5949ff341cd986777111ba1d16f" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 38913 } } }
I20260811 16:09:15.226704 13227 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:15.233557 13235 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:15.234090 13235 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 16:09:15.269040 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:38913
I20260811 16:09:15.269145 13305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:38913 every 8 connection(s)
I20260811 16:09:15.277189 8080 tablet_server.cc:178] TabletServer@127.7.228.1:38913 shutting down...
I20260811 16:09:15.277027 13307 heartbeater.cc:347] Connected to a master server at 127.7.228.62:33449
I20260811 16:09:15.281661 13307 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:15.282794 13307 heartbeater.cc:511] Master 127.7.228.62:33449 requested a full tablet report, sending...
I20260811 16:09:15.287909 12774 ts_manager.cc:194] Re-registered known tserver with Master: c91be5949ff341cd986777111ba1d16f (127.7.228.1:38913)
W20260811 16:09:15.288575 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:15.290982 12774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57724
I20260811 16:09:15.294929 13307 heartbeater.cc:503] Master 127.7.228.62:33449 was elected leader, sending a full tablet report...
W20260811 16:09:15.301438 12774 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:15.314611 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:15.315392 8080 tablet_replica.cc:330] stopping tablet replica
I20260811 16:09:15.315994 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 2 LEADER]: Raft consensus shutting down.
I20260811 16:09:15.316429 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P c91be5949ff341cd986777111ba1d16f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:15.329326 8080 tablet_server.cc:194] TabletServer@127.7.228.1:38913 shutdown complete.
I20260811 16:09:15.343582 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:09:15.366166 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:15.367494 8080 tablet_replica.cc:330] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1: stopping tablet replica
I20260811 16:09:15.368162 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 16:09:15.368664 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P b31ddf2f0ba043d283bf5b5e3dca7ad1 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 16:09:15.389214 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:09:15.399976 8080 master.cc:537] Master@127.7.228.62:33449 shutting down...
I20260811 16:09:15.417574 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:15.418143 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:15.418444 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P d2520b159ead444bbc2c680da9afad9f: stopping tablet replica
I20260811 16:09:15.437412 8080 master.cc:552] Master@127.7.228.62:33449 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1344 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260811 16:09:15.471746 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:43311
I20260811 16:09:15.473212 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:15.479703 13316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.482638 13317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.485553 13320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:15.485668 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:15.486732 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:15.486980 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:15.487155 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464555487140 us; error 0 us; skew 500 ppm
I20260811 16:09:15.487865 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:15.491016 8080 webserver.cc:460] Webserver started at http://127.7.228.62:41321/ using document root <none> and password file <none>
I20260811 16:09:15.491675 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:15.491909 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:15.492249 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:15.493860 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "38669b269ca4484b8a79afabce569639"
format_stamp: "Formatted at 2026-08-11 16:09:15 on dist-test-slave-ncp9"
I20260811 16:09:15.500262 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 16:09:15.505048 13332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:15.505918 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:09:15.506202 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "38669b269ca4484b8a79afabce569639"
format_stamp: "Formatted at 2026-08-11 16:09:15 on dist-test-slave-ncp9"
I20260811 16:09:15.506482 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:15.533912 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:15.534966 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:15.580660 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:43311
I20260811 16:09:15.580838 13428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:43311 every 8 connection(s)
I20260811 16:09:15.585094 13429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:15.595950 13429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639: Bootstrap starting.
I20260811 16:09:15.600438 13429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:15.605019 13429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639: No bootstrap required, opened a new log
I20260811 16:09:15.607024 13429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER }
I20260811 16:09:15.607452 13429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:15.607708 13429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38669b269ca4484b8a79afabce569639, State: Initialized, Role: FOLLOWER
I20260811 16:09:15.608341 13429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER }
I20260811 16:09:15.608825 13429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:15.609067 13429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:15.609324 13429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:15.613961 13429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER }
I20260811 16:09:15.614566 13429 leader_election.cc:302] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38669b269ca4484b8a79afabce569639; no voters:
I20260811 16:09:15.615970 13429 leader_election.cc:288] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:15.616322 13434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:15.618216 13434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 1 LEADER]: Becoming Leader. State: Replica: 38669b269ca4484b8a79afabce569639, State: Running, Role: LEADER
I20260811 16:09:15.619006 13434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER }
I20260811 16:09:15.619524 13429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:15.621945 13435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38669b269ca4484b8a79afabce569639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER } }
I20260811 16:09:15.622552 13435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:15.624248 13436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38669b269ca4484b8a79afabce569639. Latest consensus state: current_term: 1 leader_uuid: "38669b269ca4484b8a79afabce569639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38669b269ca4484b8a79afabce569639" member_type: VOTER } }
I20260811 16:09:15.624997 13436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:15.629993 13439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:15.635982 13439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:15.642022 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:15.647197 13439 catalog_manager.cc:1347] Generated new cluster ID: 35f47e9c143a4fbcb647bff5e1f97989
I20260811 16:09:15.647527 13439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:15.665190 13439 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:15.666949 13439 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:15.681180 13439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639: Generated new TSK 0
I20260811 16:09:15.681897 13439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:15.714524 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:15.721633 13461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.723469 13462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:15.726886 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:15.727655 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:15.728588 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:15.728785 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:15.729017 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464555728998 us; error 0 us; skew 500 ppm
I20260811 16:09:15.729592 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:15.732566 8080 webserver.cc:460] Webserver started at http://127.7.228.1:43681/ using document root <none> and password file <none>
I20260811 16:09:15.733129 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:15.733311 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:15.733570 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:15.734778 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "bf54c5c4432344fab74390cf0dbb4e9d"
format_stamp: "Formatted at 2026-08-11 16:09:15 on dist-test-slave-ncp9"
I20260811 16:09:15.740232 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 16:09:15.744266 13474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:15.745282 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:09:15.745692 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "bf54c5c4432344fab74390cf0dbb4e9d"
format_stamp: "Formatted at 2026-08-11 16:09:15 on dist-test-slave-ncp9"
I20260811 16:09:15.746093 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:15.762703 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:15.764156 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:15.776226 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:15.776547 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:15.776863 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:15.777130 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 16:09:15.848255 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:40181
I20260811 16:09:15.848383 13581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:40181 every 8 connection(s)
I20260811 16:09:15.854571 13583 heartbeater.cc:347] Connected to a master server at 127.7.228.62:43311
I20260811 16:09:15.855166 13583 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:15.856169 13583 heartbeater.cc:511] Master 127.7.228.62:43311 requested a full tablet report, sending...
I20260811 16:09:15.858817 13368 ts_manager.cc:194] Registered new tserver with Master: bf54c5c4432344fab74390cf0dbb4e9d (127.7.228.1:40181)
I20260811 16:09:15.859390 8080 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006455387s
I20260811 16:09:15.861850 13368 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46040
I20260811 16:09:15.903267 13531 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 16:09:15.904553 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:15.917239 13598 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: Bootstrap starting.
I20260811 16:09:15.921839 13598 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:15.926517 13598 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: No bootstrap required, opened a new log
I20260811 16:09:15.926976 13598 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 16:09:15.929003 13598 raft_consensus.cc:348] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } }
I20260811 16:09:15.929450 13598 raft_consensus.cc:374] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:15.929706 13598 raft_consensus.cc:729] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf54c5c4432344fab74390cf0dbb4e9d, State: Initialized, Role: FOLLOWER
I20260811 16:09:15.930231 13598 consensus_queue.cc:260] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } }
I20260811 16:09:15.930750 13598 raft_consensus.cc:388] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:15.930974 13598 raft_consensus.cc:482] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:15.931221 13598 raft_consensus.cc:3037] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:15.936533 13598 raft_consensus.cc:504] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } }
I20260811 16:09:15.937194 13598 leader_election.cc:302] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf54c5c4432344fab74390cf0dbb4e9d; no voters:
I20260811 16:09:15.938694 13598 leader_election.cc:288] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:15.939046 13600 raft_consensus.cc:2781] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:15.940960 13583 heartbeater.cc:503] Master 127.7.228.62:43311 was elected leader, sending a full tablet report...
I20260811 16:09:15.941516 13598 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 16:09:15.942431 13600 raft_consensus.cc:686] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 1 LEADER]: Becoming Leader. State: Replica: bf54c5c4432344fab74390cf0dbb4e9d, State: Running, Role: LEADER
I20260811 16:09:15.943111 13600 consensus_queue.cc:237] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } }
I20260811 16:09:15.946653 13603 tablet_replica.cc:439] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: TxnStatusTablet state changed. Reason: New leader bf54c5c4432344fab74390cf0dbb4e9d. Latest consensus state: current_term: 1 leader_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" committed_config { opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } } }
I20260811 16:09:15.947297 13603 tablet_replica.cc:442] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: This TxnStatusTablet replica's current role is: LEADER
W20260811 16:09:15.949821 13368 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:15.952668 13602 tablet_replica.cc:439] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" committed_config { opid_index: -1 peers { permanent_uuid: "bf54c5c4432344fab74390cf0dbb4e9d" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 40181 } } }
I20260811 16:09:15.953362 13602 tablet_replica.cc:442] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:15.958307 13605 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:15.958799 13605 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260811 16:09:15.959827 13368 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260811 16:09:16.017557 13500 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 16:09:16.017956 13500 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 16:09:16.018349 13500 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260811 16:09:16.027661 13500 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 16:09:16.068073 13500 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 16:09:16.068482 13500 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260811 16:09:16.076790 13500 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260811 16:09:16.089531 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.091920 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.093897 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.095804 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.097685 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.099673 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:49874
W20260811 16:09:16.111366 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
W20260811 16:09:16.113471 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
W20260811 16:09:16.115600 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
W20260811 16:09:16.117483 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
W20260811 16:09:16.119233 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
W20260811 16:09:16.121083 13531 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:49882
I20260811 16:09:16.125841 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:16.151818 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:16.152673 8080 tablet_replica.cc:330] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d: stopping tablet replica
I20260811 16:09:16.153254 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:16.153741 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P bf54c5c4432344fab74390cf0dbb4e9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:16.175037 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:16.186848 8080 master.cc:537] Master@127.7.228.62:43311 shutting down...
I20260811 16:09:16.202356 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:16.202879 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:16.203174 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 38669b269ca4484b8a79afabce569639: stopping tablet replica
I20260811 16:09:16.221242 8080 master.cc:552] Master@127.7.228.62:43311 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (775 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260811 16:09:16.244778 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:39895
I20260811 16:09:16.245919 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:16.250938 13623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:16.251966 13625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:16.259768 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:16.259778 13628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:16.261015 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:16.261222 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:16.261334 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464556261326 us; error 0 us; skew 500 ppm
I20260811 16:09:16.261806 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:16.264251 8080 webserver.cc:460] Webserver started at http://127.7.228.62:37683/ using document root <none> and password file <none>
I20260811 16:09:16.264706 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:16.264855 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:16.265138 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:16.266237 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "53e871eed0f1434c8b9d2e509df84c0e"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.271128 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 16:09:16.274783 13636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:16.275589 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 16:09:16.275887 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "53e871eed0f1434c8b9d2e509df84c0e"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.276181 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:16.288461 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:16.289564 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:16.326814 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:39895
I20260811 16:09:16.326922 13723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:39895 every 8 connection(s)
I20260811 16:09:16.330904 13726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:16.341598 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e: Bootstrap starting.
I20260811 16:09:16.345906 13726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:16.350049 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e: No bootstrap required, opened a new log
I20260811 16:09:16.351989 13726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER }
I20260811 16:09:16.352409 13726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:16.352583 13726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e871eed0f1434c8b9d2e509df84c0e, State: Initialized, Role: FOLLOWER
I20260811 16:09:16.353101 13726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER }
I20260811 16:09:16.353523 13726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:16.353699 13726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:16.353899 13726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:16.358543 13726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER }
I20260811 16:09:16.359063 13726 leader_election.cc:302] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53e871eed0f1434c8b9d2e509df84c0e; no voters:
I20260811 16:09:16.360224 13726 leader_election.cc:288] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:16.360584 13729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:16.362058 13729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 1 LEADER]: Becoming Leader. State: Replica: 53e871eed0f1434c8b9d2e509df84c0e, State: Running, Role: LEADER
I20260811 16:09:16.362699 13729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER }
I20260811 16:09:16.363267 13726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:16.366102 13730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53e871eed0f1434c8b9d2e509df84c0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER } }
I20260811 16:09:16.366191 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e871eed0f1434c8b9d2e509df84c0e. Latest consensus state: current_term: 1 leader_uuid: "53e871eed0f1434c8b9d2e509df84c0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e871eed0f1434c8b9d2e509df84c0e" member_type: VOTER } }
I20260811 16:09:16.366902 13730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:16.367026 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:16.370733 13736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:16.375873 13736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:16.378885 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:16.385097 13736 catalog_manager.cc:1347] Generated new cluster ID: 957dadf253e74323b0ade625bad62a49
I20260811 16:09:16.385427 13736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:16.408319 13736 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:16.410111 13736 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:16.426257 13736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e: Generated new TSK 0
I20260811 16:09:16.426949 13736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:16.446508 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:16.452577 13750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:16.455180 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:16.456549 13754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:16.457176 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:16.457983 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:16.458179 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:16.458340 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464556458330 us; error 0 us; skew 500 ppm
I20260811 16:09:16.458873 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:16.461274 8080 webserver.cc:460] Webserver started at http://127.7.228.1:34133/ using document root <none> and password file <none>
I20260811 16:09:16.461755 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:16.461928 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:16.462188 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:16.463311 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "a8047152c33548e2a04e978dcdb154c4"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.467960 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 16:09:16.471432 13761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:16.472322 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:09:16.472636 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "a8047152c33548e2a04e978dcdb154c4"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.472975 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:16.484598 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:16.485654 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:16.490870 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:16.491192 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:16.491482 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:16.491688 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:16.546824 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:42739
I20260811 16:09:16.546979 13877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:42739 every 8 connection(s)
I20260811 16:09:16.551576 13880 heartbeater.cc:347] Connected to a master server at 127.7.228.62:39895
I20260811 16:09:16.551963 13880 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:16.552728 13880 heartbeater.cc:511] Master 127.7.228.62:39895 requested a full tablet report, sending...
I20260811 16:09:16.554705 13669 ts_manager.cc:194] Registered new tserver with Master: a8047152c33548e2a04e978dcdb154c4 (127.7.228.1:42739)
I20260811 16:09:16.555181 8080 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.005091295s
I20260811 16:09:16.557718 13669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56494
I20260811 16:09:16.592583 13824 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 16:09:16.593829 13824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:16.605432 13897 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: Bootstrap starting.
I20260811 16:09:16.609877 13897 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:16.614151 13897 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: No bootstrap required, opened a new log
I20260811 16:09:16.614571 13897 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 16:09:16.616464 13897 raft_consensus.cc:348] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.616909 13897 raft_consensus.cc:374] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:16.617144 13897 raft_consensus.cc:729] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8047152c33548e2a04e978dcdb154c4, State: Initialized, Role: FOLLOWER
I20260811 16:09:16.617645 13897 consensus_queue.cc:260] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.618124 13897 raft_consensus.cc:388] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:16.618337 13897 raft_consensus.cc:482] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:16.618587 13897 raft_consensus.cc:3037] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:16.623337 13897 raft_consensus.cc:504] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.623905 13897 leader_election.cc:302] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8047152c33548e2a04e978dcdb154c4; no voters:
I20260811 16:09:16.625128 13897 leader_election.cc:288] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:16.625483 13901 raft_consensus.cc:2781] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:16.627097 13880 heartbeater.cc:503] Master 127.7.228.62:39895 was elected leader, sending a full tablet report...
I20260811 16:09:16.627286 13897 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: Time spent starting tablet: real 0.012s user 0.007s sys 0.007s
I20260811 16:09:16.627749 13901 raft_consensus.cc:686] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 1 LEADER]: Becoming Leader. State: Replica: a8047152c33548e2a04e978dcdb154c4, State: Running, Role: LEADER
I20260811 16:09:16.628388 13901 consensus_queue.cc:237] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.633236 13902 tablet_replica.cc:439] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8047152c33548e2a04e978dcdb154c4" committed_config { opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } } }
I20260811 16:09:16.633955 13902 tablet_replica.cc:442] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:16.634423 13903 tablet_replica.cc:439] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: TxnStatusTablet state changed. Reason: New leader a8047152c33548e2a04e978dcdb154c4. Latest consensus state: current_term: 1 leader_uuid: "a8047152c33548e2a04e978dcdb154c4" committed_config { opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } } }
I20260811 16:09:16.635077 13903 tablet_replica.cc:442] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: This TxnStatusTablet replica's current role is: LEADER
I20260811 16:09:16.635608 13913 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 16:09:16.636032 13913 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260811 16:09:16.636502 13669 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260811 16:09:16.640866 13669 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 16:09:16.666455 13824 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 16:09:16.667655 13824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:16.679502 13897 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: Bootstrap starting.
I20260811 16:09:16.683830 13897 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:16.688208 13897 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: No bootstrap required, opened a new log
I20260811 16:09:16.688666 13897 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 16:09:16.690829 13897 raft_consensus.cc:348] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.691279 13897 raft_consensus.cc:374] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:16.691555 13897 raft_consensus.cc:729] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8047152c33548e2a04e978dcdb154c4, State: Initialized, Role: FOLLOWER
I20260811 16:09:16.692112 13897 consensus_queue.cc:260] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.692610 13897 raft_consensus.cc:388] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:16.692838 13897 raft_consensus.cc:482] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:16.693099 13897 raft_consensus.cc:3037] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:16.697840 13897 raft_consensus.cc:504] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
I20260811 16:09:16.698426 13897 leader_election.cc:302] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8047152c33548e2a04e978dcdb154c4; no voters:
I20260811 16:09:16.698918 13897 leader_election.cc:288] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:16.699116 13903 raft_consensus.cc:2781] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:16.699592 13903 raft_consensus.cc:686] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 1 LEADER]: Becoming Leader. State: Replica: a8047152c33548e2a04e978dcdb154c4, State: Running, Role: LEADER
I20260811 16:09:16.700618 13897 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 16:09:16.700331 13903 consensus_queue.cc:237] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a8047152c33548e2a04e978dcdb154c4" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 42739 } }
W20260811 16:09:16.706307 13669 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260811 16:09:16.710641 13669 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260811 16:09:16.730090 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:16.749027 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:16.749773 8080 tablet_replica.cc:330] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4: stopping tablet replica
I20260811 16:09:16.750334 8080 raft_consensus.cc:2229] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:16.750779 8080 raft_consensus.cc:2258] T 22222222222222222222222222222222 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:16.753335 8080 tablet_replica.cc:330] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4: stopping tablet replica
I20260811 16:09:16.753849 8080 raft_consensus.cc:2229] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:16.754293 8080 raft_consensus.cc:2258] T 11111111111111111111111111111111 P a8047152c33548e2a04e978dcdb154c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:16.776268 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:16.788506 8080 master.cc:537] Master@127.7.228.62:39895 shutting down...
I20260811 16:09:16.805883 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:16.806456 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:16.807088 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P 53e871eed0f1434c8b9d2e509df84c0e: stopping tablet replica
I20260811 16:09:16.826880 8080 master.cc:552] Master@127.7.228.62:39895 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (606 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3692 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260811 16:09:16.851650 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:44475
I20260811 16:09:16.852720 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:16.857950 13930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:16.858765 13931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:16.861151 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:16.861433 13934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:16.862270 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:16.862460 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:16.862655 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464556862637 us; error 0 us; skew 500 ppm
I20260811 16:09:16.863204 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:16.865662 8080 webserver.cc:460] Webserver started at http://127.7.228.62:35875/ using document root <none> and password file <none>
I20260811 16:09:16.866133 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:16.866297 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:16.866554 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:16.867689 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "d29feebd5ab742219f0775528a71a5a0"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.872386 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 16:09:16.876062 13942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:16.876955 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:09:16.877250 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "d29feebd5ab742219f0775528a71a5a0"
format_stamp: "Formatted at 2026-08-11 16:09:16 on dist-test-slave-ncp9"
I20260811 16:09:16.877543 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:16.894656 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:16.895751 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:16.932507 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:44475
I20260811 16:09:16.932639 14030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:44475 every 8 connection(s)
I20260811 16:09:16.936501 14031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:16.946977 14031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0: Bootstrap starting.
I20260811 16:09:16.951372 14031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:16.955605 14031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0: No bootstrap required, opened a new log
I20260811 16:09:16.957715 14031 raft_consensus.cc:348] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER }
I20260811 16:09:16.958134 14031 raft_consensus.cc:374] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:16.958323 14031 raft_consensus.cc:729] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29feebd5ab742219f0775528a71a5a0, State: Initialized, Role: FOLLOWER
I20260811 16:09:16.958896 14031 consensus_queue.cc:260] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER }
I20260811 16:09:16.959345 14031 raft_consensus.cc:388] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:16.959542 14031 raft_consensus.cc:482] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:16.959744 14031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:16.964427 14031 raft_consensus.cc:504] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER }
I20260811 16:09:16.964959 14031 leader_election.cc:302] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d29feebd5ab742219f0775528a71a5a0; no voters:
I20260811 16:09:16.966149 14031 leader_election.cc:288] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:16.966518 14037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:16.967983 14037 raft_consensus.cc:686] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 1 LEADER]: Becoming Leader. State: Replica: d29feebd5ab742219f0775528a71a5a0, State: Running, Role: LEADER
I20260811 16:09:16.968668 14037 consensus_queue.cc:237] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER }
I20260811 16:09:16.969226 14031 sys_catalog.cc:564] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:16.971858 14038 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d29feebd5ab742219f0775528a71a5a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER } }
I20260811 16:09:16.971926 14040 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d29feebd5ab742219f0775528a71a5a0. Latest consensus state: current_term: 1 leader_uuid: "d29feebd5ab742219f0775528a71a5a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29feebd5ab742219f0775528a71a5a0" member_type: VOTER } }
I20260811 16:09:16.972854 14040 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:16.975075 14038 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:16.976341 14044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:16.981631 14044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:16.984779 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:16.990330 14044 catalog_manager.cc:1347] Generated new cluster ID: ee8dd3994b39492ca39ad37dc5b7580a
I20260811 16:09:16.990582 14044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:17.008746 14044 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:17.010612 14044 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:17.031313 14044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0: Generated new TSK 0
I20260811 16:09:17.032133 14044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:17.051913 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:17.058337 14073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:17.059132 14075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:17.061628 14080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:17.061728 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:17.062686 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:17.062870 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:17.063038 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464557063023 us; error 0 us; skew 500 ppm
I20260811 16:09:17.063575 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:17.066027 8080 webserver.cc:460] Webserver started at http://127.7.228.1:34457/ using document root <none> and password file <none>
I20260811 16:09:17.066530 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:17.066718 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:17.066974 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:17.068128 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "a0375994165c464288791f64b41bdc45"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.073372 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 16:09:17.077064 14088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.078006 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 16:09:17.078366 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "a0375994165c464288791f64b41bdc45"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.078708 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:17.093716 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:17.094805 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:17.096344 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:17.098774 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:17.098978 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.099205 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:17.099346 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.139326 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:35479
I20260811 16:09:17.139464 14206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:35479 every 8 connection(s)
I20260811 16:09:17.144124 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:17.150797 14213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:17.152395 14215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:17.156164 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:17.157379 14221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:17.158025 14208 heartbeater.cc:347] Connected to a master server at 127.7.228.62:44475
I20260811 16:09:17.158389 14208 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:17.158406 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:17.158754 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:17.158919 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464557158906 us; error 0 us; skew 500 ppm
I20260811 16:09:17.159230 14208 heartbeater.cc:511] Master 127.7.228.62:44475 requested a full tablet report, sending...
I20260811 16:09:17.159529 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:17.161511 13967 ts_manager.cc:194] Registered new tserver with Master: a0375994165c464288791f64b41bdc45 (127.7.228.1:35479)
I20260811 16:09:17.162592 8080 webserver.cc:460] Webserver started at http://127.7.228.2:44375/ using document root <none> and password file <none>
I20260811 16:09:17.163247 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:17.163496 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:17.163818 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:17.164381 13967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41342
I20260811 16:09:17.165287 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "41873dc403b9402392426659d9f82c33"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.170047 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 16:09:17.173709 14230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.174520 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 16:09:17.174821 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "41873dc403b9402392426659d9f82c33"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.175107 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:17.198568 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:17.199617 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:17.201133 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:17.203435 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:17.203641 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.203869 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:17.204016 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.242798 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:45377
I20260811 16:09:17.242899 14333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:45377 every 8 connection(s)
I20260811 16:09:17.247527 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:17.253871 14339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:17.256347 14340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:17.260475 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:17.260521 14342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:17.261745 14335 heartbeater.cc:347] Connected to a master server at 127.7.228.62:44475
I20260811 16:09:17.261937 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260811 16:09:17.262195 14335 heartbeater.cc:464] Registering TS with master...
W20260811 16:09:17.262212 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:17.262632 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464557262611 us; error 0 us; skew 500 ppm
I20260811 16:09:17.263203 14335 heartbeater.cc:511] Master 127.7.228.62:44475 requested a full tablet report, sending...
I20260811 16:09:17.263427 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:17.265506 13967 ts_manager.cc:194] Registered new tserver with Master: 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377)
I20260811 16:09:17.266618 8080 webserver.cc:460] Webserver started at http://127.7.228.3:42059/ using document root <none> and password file <none>
I20260811 16:09:17.267125 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:17.267320 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:17.267385 13967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41348
I20260811 16:09:17.267567 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:17.268872 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-2-root/instance:
uuid: "e64526e17ee146e7be05658ac5e7dd2b"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.274233 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 16:09:17.277967 14349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.278848 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:09:17.279203 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-2-root
uuid: "e64526e17ee146e7be05658ac5e7dd2b"
format_stamp: "Formatted at 2026-08-11 16:09:17 on dist-test-slave-ncp9"
I20260811 16:09:17.279553 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:17.300042 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:17.301134 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:17.302558 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:17.304790 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:17.305004 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.305239 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:17.305384 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:17.343384 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.3:41319
I20260811 16:09:17.343537 14453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.3:41319 every 8 connection(s)
I20260811 16:09:17.359124 14454 heartbeater.cc:347] Connected to a master server at 127.7.228.62:44475
I20260811 16:09:17.359586 14454 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:17.360309 14454 heartbeater.cc:511] Master 127.7.228.62:44475 requested a full tablet report, sending...
I20260811 16:09:17.362435 8080 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016289067s
I20260811 16:09:17.362318 13967 ts_manager.cc:194] Registered new tserver with Master: e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3:41319)
I20260811 16:09:17.364498 13967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41354
I20260811 16:09:17.386538 8080 test_util.cc:268] Using random seed: -1319580912
I20260811 16:09:17.412501 13967 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41370:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 16:09:17.459012 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 (DEFAULT_TABLE table=test-workload [id=c4ad7f10ab5546f3a9854dd20f7e2905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:17.460469 14400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0fb6a0979a43e49ab9cf933e14ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:17.476655 14281 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 (DEFAULT_TABLE table=test-workload [id=c4ad7f10ab5546f3a9854dd20f7e2905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:17.478183 14281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0fb6a0979a43e49ab9cf933e14ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:17.478641 14137 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 (DEFAULT_TABLE table=test-workload [id=c4ad7f10ab5546f3a9854dd20f7e2905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:17.479825 14137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0fb6a0979a43e49ab9cf933e14ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:17.490057 14486 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Bootstrap starting.
I20260811 16:09:17.496943 14487 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Bootstrap starting.
I20260811 16:09:17.497618 14486 tablet_bootstrap.cc:654] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:17.502197 14488 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: Bootstrap starting.
I20260811 16:09:17.502275 14487 tablet_bootstrap.cc:654] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:17.504276 14486 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: No bootstrap required, opened a new log
I20260811 16:09:17.506196 14486 ts_tablet_manager.cc:1397] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260811 16:09:17.507946 14488 tablet_bootstrap.cc:654] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:17.508141 14487 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: No bootstrap required, opened a new log
I20260811 16:09:17.508647 14487 ts_tablet_manager.cc:1397] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 16:09:17.509054 14486 raft_consensus.cc:348] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.509879 14486 raft_consensus.cc:374] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:17.510176 14486 raft_consensus.cc:729] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64526e17ee146e7be05658ac5e7dd2b, State: Initialized, Role: FOLLOWER
I20260811 16:09:17.510944 14486 consensus_queue.cc:260] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.511508 14487 raft_consensus.cc:348] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.512315 14487 raft_consensus.cc:374] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:17.512638 14487 raft_consensus.cc:729] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41873dc403b9402392426659d9f82c33, State: Initialized, Role: FOLLOWER
I20260811 16:09:17.512858 14488 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: No bootstrap required, opened a new log
I20260811 16:09:17.513621 14488 ts_tablet_manager.cc:1397] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260811 16:09:17.513878 14454 heartbeater.cc:503] Master 127.7.228.62:44475 was elected leader, sending a full tablet report...
I20260811 16:09:17.513450 14487 consensus_queue.cc:260] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.514153 14486 ts_tablet_manager.cc:1428] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 16:09:17.516378 14335 heartbeater.cc:503] Master 127.7.228.62:44475 was elected leader, sending a full tablet report...
I20260811 16:09:17.516582 14488 raft_consensus.cc:348] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.517431 14488 raft_consensus.cc:374] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:17.517645 14487 ts_tablet_manager.cc:1428] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 16:09:17.517755 14488 raft_consensus.cc:729] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0375994165c464288791f64b41bdc45, State: Initialized, Role: FOLLOWER
I20260811 16:09:17.518852 14488 consensus_queue.cc:260] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.521875 14208 heartbeater.cc:503] Master 127.7.228.62:44475 was elected leader, sending a full tablet report...
I20260811 16:09:17.523633 14488 ts_tablet_manager.cc:1428] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: Time spent starting tablet: real 0.010s user 0.003s sys 0.005s
I20260811 16:09:17.736999 14492 raft_consensus.cc:482] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:09:17.737577 14492 raft_consensus.cc:504] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.739535 14492 leader_election.cc:288] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377), a0375994165c464288791f64b41bdc45 (127.7.228.1:35479)
I20260811 16:09:17.753154 14293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0fb6a0979a43e49ab9cf933e14ec17" candidate_uuid: "e64526e17ee146e7be05658ac5e7dd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41873dc403b9402392426659d9f82c33" is_pre_election: true
I20260811 16:09:17.754007 14293 raft_consensus.cc:2454] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64526e17ee146e7be05658ac5e7dd2b in term 0.
I20260811 16:09:17.754242 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0fb6a0979a43e49ab9cf933e14ec17" candidate_uuid: "e64526e17ee146e7be05658ac5e7dd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0375994165c464288791f64b41bdc45" is_pre_election: true
I20260811 16:09:17.754964 14158 raft_consensus.cc:2454] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e64526e17ee146e7be05658ac5e7dd2b in term 0.
I20260811 16:09:17.755264 14354 leader_election.cc:302] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41873dc403b9402392426659d9f82c33, e64526e17ee146e7be05658ac5e7dd2b; no voters:
I20260811 16:09:17.756058 14492 raft_consensus.cc:2781] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:09:17.756321 14492 raft_consensus.cc:482] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:09:17.756582 14492 raft_consensus.cc:3037] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:17.761475 14492 raft_consensus.cc:504] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.763175 14492 leader_election.cc:288] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [CANDIDATE]: Term 1 election: Requested vote from peers 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377), a0375994165c464288791f64b41bdc45 (127.7.228.1:35479)
I20260811 16:09:17.764081 14293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0fb6a0979a43e49ab9cf933e14ec17" candidate_uuid: "e64526e17ee146e7be05658ac5e7dd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41873dc403b9402392426659d9f82c33"
I20260811 16:09:17.764214 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0fb6a0979a43e49ab9cf933e14ec17" candidate_uuid: "e64526e17ee146e7be05658ac5e7dd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0375994165c464288791f64b41bdc45"
I20260811 16:09:17.764596 14293 raft_consensus.cc:3037] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:17.764780 14158 raft_consensus.cc:3037] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:17.769796 14158 raft_consensus.cc:2454] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64526e17ee146e7be05658ac5e7dd2b in term 1.
I20260811 16:09:17.769796 14293 raft_consensus.cc:2454] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e64526e17ee146e7be05658ac5e7dd2b in term 1.
I20260811 16:09:17.771107 14350 leader_election.cc:302] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0375994165c464288791f64b41bdc45, e64526e17ee146e7be05658ac5e7dd2b; no voters:
I20260811 16:09:17.771829 14492 raft_consensus.cc:2781] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:17.772722 14492 raft_consensus.cc:686] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 LEADER]: Becoming Leader. State: Replica: e64526e17ee146e7be05658ac5e7dd2b, State: Running, Role: LEADER
I20260811 16:09:17.773535 14492 consensus_queue.cc:237] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:17.780418 13967 catalog_manager.cc:5495] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b reported cstate change: term changed from 0 to 1, leader changed from <none> to e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3). New cstate: current_term: 1 leader_uuid: "e64526e17ee146e7be05658ac5e7dd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } }
I20260811 16:09:18.103327 14281 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:18.114771 14137 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:18.127481 14400 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:18.264201 14492 consensus_queue.cc:1035] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 16:09:18.293766 14555 consensus_queue.cc:1035] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d29feebd5ab742219f0775528a71a5a0 | 127.7.228.62:44475 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
41873dc403b9402392426659d9f82c33 | 127.7.228.2:45377 | HEALTHY | <none> | 0 | 0
a0375994165c464288791f64b41bdc45 | 127.7.228.1:35479 | HEALTHY | <none> | 0 | 0
e64526e17ee146e7be05658ac5e7dd2b | 127.7.228.3:41319 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 0e0fb6a0979a43e49ab9cf933e14ec17 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3:41319): RUNNING [LEADER]
41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): RUNNING
a0375994165c464288791f64b41bdc45 (127.7.228.1:35479): RUNNING
All reported replicas are:
A = e64526e17ee146e7be05658ac5e7dd2b
B = 41873dc403b9402392426659d9f82c33
C = a0375994165c464288791f64b41bdc45
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 16:09:18.340664 8080 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 16:09:18.709064 14281 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:18.722069 14400 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:18.726583 14137 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d29feebd5ab742219f0775528a71a5a0 | 127.7.228.62:44475 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
41873dc403b9402392426659d9f82c33 | 127.7.228.2:45377 | HEALTHY | <none> | 0 | 0
a0375994165c464288791f64b41bdc45 | 127.7.228.1:35479 | HEALTHY | <none> | 0 | 0
e64526e17ee146e7be05658ac5e7dd2b | 127.7.228.3:41319 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 16:09:18.920229 8080 log_verifier.cc:126] Checking tablet 0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:18.983712 8080 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:18.984267 8080 tablet_replica.cc:330] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: stopping tablet replica
I20260811 16:09:18.984740 8080 raft_consensus.cc:2229] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:18.985226 8080 raft_consensus.cc:2258] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:19.279516 14281 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:19.284195 14137 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:19.285379 14400 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20260811 16:09:19.324966 14354 consensus_peers.cc:487] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b -> Peer 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): Couldn't send request to peer 41873dc403b9402392426659d9f82c33. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:19.326323 14555 raft_consensus.cc:998] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Attempting to remove follower 41873dc403b9402392426659d9f82c33 from the Raft config. Reason: The tablet replica hosted on peer 41873dc403b9402392426659d9f82c33 has failed
I20260811 16:09:19.329627 14555 consensus_queue.cc:237] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } }
I20260811 16:09:19.336609 14158 raft_consensus.cc:1261] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Refusing update from remote peer e64526e17ee146e7be05658ac5e7dd2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 16:09:19.338267 14555 consensus_queue.cc:1035] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:09:19.358827 14644 raft_consensus.cc:2932] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } }
I20260811 16:09:19.361076 14156 raft_consensus.cc:2932] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } }
I20260811 16:09:19.379861 13960 catalog_manager.cc:5495] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b reported cstate change: config changed from index -1 to 2, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) evicted. New cstate: current_term: 1 leader_uuid: "e64526e17ee146e7be05658ac5e7dd2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } }
I20260811 16:09:19.432051 14415 consensus_queue.cc:237] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } }
I20260811 16:09:19.434432 14281 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 with delete_type TABLET_DATA_TOMBSTONED (TS 41873dc403b9402392426659d9f82c33 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:41710
I20260811 16:09:19.446478 14657 ts_tablet_manager.cc:1905] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260811 16:09:19.449432 14354 consensus_peers.cc:487] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b -> Peer 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): Couldn't send request to peer 41873dc403b9402392426659d9f82c33. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:19.451025 14156 raft_consensus.cc:1261] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Refusing update from remote peer e64526e17ee146e7be05658ac5e7dd2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260811 16:09:19.452862 14555 consensus_queue.cc:1035] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:09:19.460601 14644 raft_consensus.cc:2932] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } }
I20260811 16:09:19.471239 14156 raft_consensus.cc:2932] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } }
I20260811 16:09:19.484419 13946 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260811 16:09:19.487671 14657 ts_tablet_manager.cc:1918] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260811 16:09:19.488420 14657 log.cc:1196] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786464498564520-8080-0/minicluster-data/ts-1-root/wals/0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:19.490923 13945 catalog_manager.cc:4841] TS 41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): tablet 0e0fb6a0979a43e49ab9cf933e14ec17 (table test-workload [id=c4ad7f10ab5546f3a9854dd20f7e2905]) successfully deleted
I20260811 16:09:19.493974 13960 catalog_manager.cc:5495] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b reported cstate change: config changed from index 2 to 3, VOTER 41873dc403b9402392426659d9f82c33 (127.7.228.2) added. New cstate: current_term: 1 leader_uuid: "e64526e17ee146e7be05658ac5e7dd2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } } }
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d29feebd5ab742219f0775528a71a5a0 | 127.7.228.62:44475 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
41873dc403b9402392426659d9f82c33 | 127.7.228.2:45377 | HEALTHY | <none> | 0 | 0
a0375994165c464288791f64b41bdc45 | 127.7.228.1:35479 | HEALTHY | <none> | 0 | 0
e64526e17ee146e7be05658ac5e7dd2b | 127.7.228.3:41319 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 0e0fb6a0979a43e49ab9cf933e14ec17 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3:41319): RUNNING [LEADER]
41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
a0375994165c464288791f64b41bdc45 (127.7.228.1:35479): RUNNING
All reported replicas are:
A = e64526e17ee146e7be05658ac5e7dd2b
B = 41873dc403b9402392426659d9f82c33
C = a0375994165c464288791f64b41bdc45
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | [config not available] | | |
C | A* B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 16:09:19.563263 8080 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 16:09:19.924906 14694 ts_tablet_manager.cc:927] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Initiating tablet copy from peer e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3:41319)
I20260811 16:09:19.936031 14694 tablet_copy_client.cc:316] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet copy: Beginning tablet copy session from remote peer at address 127.7.228.3:41319
I20260811 16:09:19.957998 14281 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:19.962459 14137 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:19.963936 14400 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:19.967020 14433 tablet_copy_service.cc:140] P e64526e17ee146e7be05658ac5e7dd2b: Received BeginTabletCopySession request for tablet 0e0fb6a0979a43e49ab9cf933e14ec17 from peer 41873dc403b9402392426659d9f82c33 ({username='slave'} at 127.0.0.1:46372)
I20260811 16:09:19.967643 14433 tablet_copy_service.cc:161] P e64526e17ee146e7be05658ac5e7dd2b: Beginning new tablet copy session on tablet 0e0fb6a0979a43e49ab9cf933e14ec17 from peer 41873dc403b9402392426659d9f82c33 at {username='slave'} at 127.0.0.1:46372: session id = 41873dc403b9402392426659d9f82c33-0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:19.974514 14433 tablet_copy_source_session.cc:205] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 16:09:19.977258 14694 ts_tablet_manager.cc:1905] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260811 16:09:19.990350 14694 ts_tablet_manager.cc:1918] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260811 16:09:19.991009 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0fb6a0979a43e49ab9cf933e14ec17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:19.997455 14694 tablet_copy_client.cc:791] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet copy: Starting download of 0 data blocks...
I20260811 16:09:19.998024 14694 tablet_copy_client.cc:655] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet copy: Starting download of 1 WAL segments...
I20260811 16:09:20.001946 14694 tablet_copy_client.cc:531] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 16:09:20.011829 14694 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Bootstrap starting.
I20260811 16:09:20.050690 14694 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:09:20.051630 14694 tablet_bootstrap.cc:492] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Bootstrap complete.
I20260811 16:09:20.052354 14694 ts_tablet_manager.cc:1397] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Time spent bootstrapping tablet: real 0.041s user 0.015s sys 0.024s
I20260811 16:09:20.055140 14694 raft_consensus.cc:348] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } }
I20260811 16:09:20.055912 14694 raft_consensus.cc:729] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41873dc403b9402392426659d9f82c33, State: Initialized, Role: FOLLOWER
I20260811 16:09:20.056665 14694 consensus_queue.cc:260] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e64526e17ee146e7be05658ac5e7dd2b" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 41319 } } peers { permanent_uuid: "a0375994165c464288791f64b41bdc45" member_type: VOTER last_known_addr { host: "127.7.228.1" port: 35479 } } peers { permanent_uuid: "41873dc403b9402392426659d9f82c33" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 45377 } }
I20260811 16:09:20.071245 14694 ts_tablet_manager.cc:1428] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: Time spent starting tablet: real 0.019s user 0.014s sys 0.007s
I20260811 16:09:20.073211 14433 tablet_copy_service.cc:342] P e64526e17ee146e7be05658ac5e7dd2b: Request end of tablet copy session 41873dc403b9402392426659d9f82c33-0e0fb6a0979a43e49ab9cf933e14ec17 received from {username='slave'} at 127.0.0.1:46372
I20260811 16:09:20.073654 14433 tablet_copy_service.cc:433] P e64526e17ee146e7be05658ac5e7dd2b: ending tablet copy session 41873dc403b9402392426659d9f82c33-0e0fb6a0979a43e49ab9cf933e14ec17 on tablet 0e0fb6a0979a43e49ab9cf933e14ec17 with peer 41873dc403b9402392426659d9f82c33
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d29feebd5ab742219f0775528a71a5a0 | 127.7.228.62:44475 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
41873dc403b9402392426659d9f82c33 | 127.7.228.2:45377 | HEALTHY | <none> | 0 | 0
a0375994165c464288791f64b41bdc45 | 127.7.228.1:35479 | HEALTHY | <none> | 0 | 0
e64526e17ee146e7be05658ac5e7dd2b | 127.7.228.3:41319 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 0e0fb6a0979a43e49ab9cf933e14ec17 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
e64526e17ee146e7be05658ac5e7dd2b (127.7.228.3:41319): RUNNING [LEADER]
a0375994165c464288791f64b41bdc45 (127.7.228.1:35479): RUNNING
41873dc403b9402392426659d9f82c33 (127.7.228.2:45377): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = e64526e17ee146e7be05658ac5e7dd2b
B = a0375994165c464288791f64b41bdc45
C = 41873dc403b9402392426659d9f82c33
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 3 | Yes
B | A* B C | 1 | 3 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 16:09:20.187079 8080 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 16:09:20.416854 14293 raft_consensus.cc:1203] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3] Dedup: 1.3->[]
I20260811 16:09:20.638577 14281 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:20.650812 14137 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:20.656224 14400 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d29feebd5ab742219f0775528a71a5a0 | 127.7.228.62:44475 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
41873dc403b9402392426659d9f82c33 | 127.7.228.2:45377 | HEALTHY | <none> | 0 | 0
a0375994165c464288791f64b41bdc45 | 127.7.228.1:35479 | HEALTHY | <none> | 0 | 0
e64526e17ee146e7be05658ac5e7dd2b | 127.7.228.3:41319 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 16:09:20.837463 8080 log_verifier.cc:126] Checking tablet 0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:20.899382 8080 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 0e0fb6a0979a43e49ab9cf933e14ec17
I20260811 16:09:20.903910 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:20.933485 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:20.934314 8080 tablet_replica.cc:330] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45: stopping tablet replica
I20260811 16:09:20.935007 8080 raft_consensus.cc:2229] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:20.935527 8080 raft_consensus.cc:2258] T 0e0fb6a0979a43e49ab9cf933e14ec17 P a0375994165c464288791f64b41bdc45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:20.956581 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:20.968019 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:09:20.991675 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:20.992403 8080 tablet_replica.cc:330] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33: stopping tablet replica
I20260811 16:09:20.992969 8080 raft_consensus.cc:2229] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:20.993366 8080 raft_consensus.cc:2258] T 0e0fb6a0979a43e49ab9cf933e14ec17 P 41873dc403b9402392426659d9f82c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:21.015131 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:09:21.026031 8080 tablet_server.cc:178] TabletServer@127.7.228.3:0 shutting down...
I20260811 16:09:21.045640 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:21.046298 8080 tablet_replica.cc:330] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b: stopping tablet replica
I20260811 16:09:21.046802 8080 raft_consensus.cc:2229] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:21.047587 8080 raft_consensus.cc:2258] T 0e0fb6a0979a43e49ab9cf933e14ec17 P e64526e17ee146e7be05658ac5e7dd2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:21.067379 8080 tablet_server.cc:194] TabletServer@127.7.228.3:0 shutdown complete.
I20260811 16:09:21.078526 8080 master.cc:537] Master@127.7.228.62:44475 shutting down...
I20260811 16:09:21.100406 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:21.101020 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:21.101390 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P d29feebd5ab742219f0775528a71a5a0: stopping tablet replica
I20260811 16:09:21.120415 8080 master.cc:552] Master@127.7.228.62:44475 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (4301 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260811 16:09:21.153599 8080 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.228.62:45977
I20260811 16:09:21.154667 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:21.160094 14751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.160740 14753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.162777 14755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.163852 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:21.164570 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:21.164749 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:21.164911 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464561164873 us; error 0 us; skew 500 ppm
I20260811 16:09:21.165413 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:21.167856 8080 webserver.cc:460] Webserver started at http://127.7.228.62:37209/ using document root <none> and password file <none>
I20260811 16:09:21.168334 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:21.168499 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:21.168780 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:21.169968 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/master-0-root/instance:
uuid: "ba9d28588c2f42059fac1fd2d2eba91c"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.174741 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 16:09:21.178298 14772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.179178 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 16:09:21.179443 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/master-0-root
uuid: "ba9d28588c2f42059fac1fd2d2eba91c"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.179725 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:21.211230 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:21.212459 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:21.252784 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.62:45977
I20260811 16:09:21.252949 14845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.62:45977 every 8 connection(s)
I20260811 16:09:21.256735 14846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:21.268271 14846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c: Bootstrap starting.
I20260811 16:09:21.273093 14846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:21.277912 14846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c: No bootstrap required, opened a new log
I20260811 16:09:21.280185 14846 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER }
I20260811 16:09:21.280668 14846 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:21.280931 14846 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9d28588c2f42059fac1fd2d2eba91c, State: Initialized, Role: FOLLOWER
I20260811 16:09:21.281612 14846 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER }
I20260811 16:09:21.282104 14846 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 16:09:21.282346 14846 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 16:09:21.282599 14846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:21.287516 14846 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER }
I20260811 16:09:21.288081 14846 leader_election.cc:302] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba9d28588c2f42059fac1fd2d2eba91c; no voters:
I20260811 16:09:21.289310 14846 leader_election.cc:288] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 16:09:21.289749 14850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:21.291018 14850 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 1 LEADER]: Becoming Leader. State: Replica: ba9d28588c2f42059fac1fd2d2eba91c, State: Running, Role: LEADER
I20260811 16:09:21.291689 14850 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER }
I20260811 16:09:21.292235 14846 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 16:09:21.296188 14851 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER } }
I20260811 16:09:21.296958 14851 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:21.299834 14853 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba9d28588c2f42059fac1fd2d2eba91c. Latest consensus state: current_term: 1 leader_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9d28588c2f42059fac1fd2d2eba91c" member_type: VOTER } }
I20260811 16:09:21.300588 14853 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [sys.catalog]: This master's current role is: LEADER
I20260811 16:09:21.303509 14861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 16:09:21.308470 14861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 16:09:21.309649 8080 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 16:09:21.317123 14861 catalog_manager.cc:1347] Generated new cluster ID: 8907b0aa06294d1dbe9ce19737447c12
I20260811 16:09:21.317396 14861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 16:09:21.332183 14861 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 16:09:21.334070 14861 catalog_manager.cc:1504] Loading token signing keys...
I20260811 16:09:21.355281 14861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c: Generated new TSK 0
I20260811 16:09:21.355981 14861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 16:09:21.376591 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:21.383164 14876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.384109 14878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.385831 14880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.386214 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:21.387133 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:21.387320 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:21.387437 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464561387428 us; error 0 us; skew 500 ppm
I20260811 16:09:21.387964 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:21.390650 8080 webserver.cc:460] Webserver started at http://127.7.228.1:37805/ using document root <none> and password file <none>
I20260811 16:09:21.391170 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:21.391338 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:21.391568 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:21.392760 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-0-root/instance:
uuid: "9d336df8967d4736aa84ee6e4d26e969"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.397636 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 16:09:21.401273 14890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.402141 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 16:09:21.402437 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-0-root
uuid: "9d336df8967d4736aa84ee6e4d26e969"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.402742 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:21.423484 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:21.424736 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:21.426339 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:21.428973 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:21.429179 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.429450 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:21.429592 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.477861 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.1:45239
I20260811 16:09:21.477998 14995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.1:45239 every 8 connection(s)
I20260811 16:09:21.482704 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:21.490450 15002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.491426 15003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.498770 15005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.499902 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:21.500171 14997 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45977
I20260811 16:09:21.500599 14997 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:21.500986 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:21.501210 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:21.501389 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464561501376 us; error 0 us; skew 500 ppm
I20260811 16:09:21.501348 14997 heartbeater.cc:511] Master 127.7.228.62:45977 requested a full tablet report, sending...
I20260811 16:09:21.502133 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:21.503919 14794 ts_manager.cc:194] Registered new tserver with Master: 9d336df8967d4736aa84ee6e4d26e969 (127.7.228.1:45239)
I20260811 16:09:21.505728 8080 webserver.cc:460] Webserver started at http://127.7.228.2:40943/ using document root <none> and password file <none>
I20260811 16:09:21.506266 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:21.506454 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:21.506518 14794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56814
I20260811 16:09:21.506704 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:21.508106 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-1-root/instance:
uuid: "3e7c890486f846d495db1b41ab40a640"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.513150 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 16:09:21.517005 15012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.517853 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 16:09:21.518154 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-1-root
uuid: "3e7c890486f846d495db1b41ab40a640"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.518442 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:21.540623 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:21.542014 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:21.543537 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:21.545957 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:21.546190 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.546408 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:21.546547 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.588858 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.2:46297
I20260811 16:09:21.589078 15117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.2:46297 every 8 connection(s)
I20260811 16:09:21.593804 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:21.601280 15121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.602748 15122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.605217 15125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.605693 8080 server_base.cc:1034] running on GCE node
I20260811 16:09:21.607131 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:21.607384 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:21.607560 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464561607548 us; error 0 us; skew 500 ppm
I20260811 16:09:21.608289 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:21.610514 15118 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45977
I20260811 16:09:21.610930 15118 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:21.611261 8080 webserver.cc:460] Webserver started at http://127.7.228.3:41239/ using document root <none> and password file <none>
I20260811 16:09:21.611768 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:21.611743 15118 heartbeater.cc:511] Master 127.7.228.62:45977 requested a full tablet report, sending...
I20260811 16:09:21.612183 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:21.612514 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:21.613889 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root/instance:
uuid: "a924b46d9a864a67a9c3a8812500540a"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.614223 14794 ts_manager.cc:194] Registered new tserver with Master: 3e7c890486f846d495db1b41ab40a640 (127.7.228.2:46297)
I20260811 16:09:21.615942 14794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56820
I20260811 16:09:21.619068 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 16:09:21.623003 15134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.623792 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 16:09:21.624097 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root
uuid: "a924b46d9a864a67a9c3a8812500540a"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.624388 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:21.640789 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:21.642086 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:21.643590 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:21.645973 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:21.646183 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.646401 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:21.646543 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.686810 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.3:35927
I20260811 16:09:21.686933 15236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.3:35927 every 8 connection(s)
I20260811 16:09:21.691634 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 16:09:21.698810 15241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 16:09:21.699879 15243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.701989 8080 server_base.cc:1034] running on GCE node
W20260811 16:09:21.702708 15246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 16:09:21.703817 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 16:09:21.704105 8080 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 16:09:21.704315 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786464561704299 us; error 0 us; skew 500 ppm
I20260811 16:09:21.705057 8080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 16:09:21.705621 15237 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45977
I20260811 16:09:21.706058 15237 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:21.706897 15237 heartbeater.cc:511] Master 127.7.228.62:45977 requested a full tablet report, sending...
I20260811 16:09:21.707896 8080 webserver.cc:460] Webserver started at http://127.7.228.4:34029/ using document root <none> and password file <none>
I20260811 16:09:21.708437 8080 fs_manager.cc:359] Metadata directory not provided
I20260811 16:09:21.708630 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 16:09:21.708877 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 16:09:21.708990 14794 ts_manager.cc:194] Registered new tserver with Master: a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927)
I20260811 16:09:21.710071 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-3-root/instance:
uuid: "a8620486dc7c4c92b88a816c9f29cdfd"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.710770 14794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56822
I20260811 16:09:21.714928 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 16:09:21.718874 15255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.719712 8080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 16:09:21.720027 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-3-root
uuid: "a8620486dc7c4c92b88a816c9f29cdfd"
format_stamp: "Formatted at 2026-08-11 16:09:21 on dist-test-slave-ncp9"
I20260811 16:09:21.720319 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 16:09:21.746294 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 16:09:21.747632 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 16:09:21.749142 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 16:09:21.751684 8080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 16:09:21.751894 8080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.752117 8080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 16:09:21.752264 8080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 16:09:21.793325 8080 rpc_server.cc:307] RPC server started. Bound to: 127.7.228.4:33275
I20260811 16:09:21.793423 15360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.228.4:33275 every 8 connection(s)
I20260811 16:09:21.809451 15361 heartbeater.cc:347] Connected to a master server at 127.7.228.62:45977
I20260811 16:09:21.809845 15361 heartbeater.cc:464] Registering TS with master...
I20260811 16:09:21.810840 15361 heartbeater.cc:511] Master 127.7.228.62:45977 requested a full tablet report, sending...
I20260811 16:09:21.812711 14794 ts_manager.cc:194] Registered new tserver with Master: a8620486dc7c4c92b88a816c9f29cdfd (127.7.228.4:33275)
I20260811 16:09:21.812846 8080 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016517022s
I20260811 16:09:21.814541 14794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56836
I20260811 16:09:21.835716 8080 test_util.cc:268] Using random seed: -1315131734
I20260811 16:09:21.860849 14794 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56862:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 16:09:21.906270 15303 tablet_service.cc:1467] Processing CreateTablet for tablet c93c8c092e6c4941a662e44c99eaa583 (DEFAULT_TABLE table=test-workload [id=2972837391624076ba8435f9cb8e10b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:21.907606 15303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93c8c092e6c4941a662e44c99eaa583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:21.914558 15177 tablet_service.cc:1467] Processing CreateTablet for tablet c93c8c092e6c4941a662e44c99eaa583 (DEFAULT_TABLE table=test-workload [id=2972837391624076ba8435f9cb8e10b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:21.914580 15058 tablet_service.cc:1467] Processing CreateTablet for tablet c93c8c092e6c4941a662e44c99eaa583 (DEFAULT_TABLE table=test-workload [id=2972837391624076ba8435f9cb8e10b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 16:09:21.916132 15058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93c8c092e6c4941a662e44c99eaa583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:21.916126 15177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93c8c092e6c4941a662e44c99eaa583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:21.922206 15401 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: Bootstrap starting.
I20260811 16:09:21.928345 15401 tablet_bootstrap.cc:654] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:21.932730 15401 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: No bootstrap required, opened a new log
I20260811 16:09:21.933212 15401 ts_tablet_manager.cc:1397] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.002s
I20260811 16:09:21.934559 15404 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: Bootstrap starting.
I20260811 16:09:21.936041 15401 raft_consensus.cc:348] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.936998 15401 raft_consensus.cc:374] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:21.937295 15401 raft_consensus.cc:729] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8620486dc7c4c92b88a816c9f29cdfd, State: Initialized, Role: FOLLOWER
I20260811 16:09:21.938141 15401 consensus_queue.cc:260] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.941141 15361 heartbeater.cc:503] Master 127.7.228.62:45977 was elected leader, sending a full tablet report...
I20260811 16:09:21.941572 15401 ts_tablet_manager.cc:1428] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260811 16:09:21.942299 15404 tablet_bootstrap.cc:654] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:21.944808 15406 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Bootstrap starting.
I20260811 16:09:21.948626 15404 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: No bootstrap required, opened a new log
I20260811 16:09:21.949208 15404 ts_tablet_manager.cc:1397] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260811 16:09:21.951882 15404 raft_consensus.cc:348] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.952466 15404 raft_consensus.cc:374] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:21.952697 15404 raft_consensus.cc:729] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e7c890486f846d495db1b41ab40a640, State: Initialized, Role: FOLLOWER
I20260811 16:09:21.953130 15406 tablet_bootstrap.cc:654] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Neither blocks nor log segments found. Creating new log.
I20260811 16:09:21.953339 15404 consensus_queue.cc:260] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.956075 15118 heartbeater.cc:503] Master 127.7.228.62:45977 was elected leader, sending a full tablet report...
I20260811 16:09:21.956504 15404 ts_tablet_manager.cc:1428] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 16:09:21.958282 15406 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: No bootstrap required, opened a new log
I20260811 16:09:21.958748 15406 ts_tablet_manager.cc:1397] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260811 16:09:21.961689 15406 raft_consensus.cc:348] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.962834 15406 raft_consensus.cc:374] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 16:09:21.963119 15406 raft_consensus.cc:729] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a924b46d9a864a67a9c3a8812500540a, State: Initialized, Role: FOLLOWER
I20260811 16:09:21.963747 15406 consensus_queue.cc:260] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:21.966104 15237 heartbeater.cc:503] Master 127.7.228.62:45977 was elected leader, sending a full tablet report...
I20260811 16:09:21.966645 15406 ts_tablet_manager.cc:1428] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 16:09:22.185883 15412 raft_consensus.cc:482] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 16:09:22.186395 15412 raft_consensus.cc:504] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:22.188403 15412 leader_election.cc:288] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8620486dc7c4c92b88a816c9f29cdfd (127.7.228.4:33275), a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927)
I20260811 16:09:22.202804 15318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93c8c092e6c4941a662e44c99eaa583" candidate_uuid: "3e7c890486f846d495db1b41ab40a640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" is_pre_election: true
I20260811 16:09:22.203676 15318 raft_consensus.cc:2454] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e7c890486f846d495db1b41ab40a640 in term 0.
I20260811 16:09:22.204834 15015 leader_election.cc:302] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e7c890486f846d495db1b41ab40a640, a8620486dc7c4c92b88a816c9f29cdfd; no voters:
I20260811 16:09:22.205672 15412 raft_consensus.cc:2781] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 16:09:22.206048 15412 raft_consensus.cc:482] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 16:09:22.206354 15412 raft_consensus.cc:3037] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:22.206561 15192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93c8c092e6c4941a662e44c99eaa583" candidate_uuid: "3e7c890486f846d495db1b41ab40a640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a924b46d9a864a67a9c3a8812500540a" is_pre_election: true
I20260811 16:09:22.207183 15192 raft_consensus.cc:2454] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e7c890486f846d495db1b41ab40a640 in term 0.
I20260811 16:09:22.211598 15412 raft_consensus.cc:504] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:22.213326 15412 leader_election.cc:288] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [CANDIDATE]: Term 1 election: Requested vote from peers a8620486dc7c4c92b88a816c9f29cdfd (127.7.228.4:33275), a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927)
I20260811 16:09:22.214051 15318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93c8c092e6c4941a662e44c99eaa583" candidate_uuid: "3e7c890486f846d495db1b41ab40a640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8620486dc7c4c92b88a816c9f29cdfd"
I20260811 16:09:22.214435 15192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93c8c092e6c4941a662e44c99eaa583" candidate_uuid: "3e7c890486f846d495db1b41ab40a640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a924b46d9a864a67a9c3a8812500540a"
I20260811 16:09:22.214777 15318 raft_consensus.cc:3037] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:22.215036 15192 raft_consensus.cc:3037] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 0 FOLLOWER]: Advancing to term 1
I20260811 16:09:22.222114 15192 raft_consensus.cc:2454] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e7c890486f846d495db1b41ab40a640 in term 1.
I20260811 16:09:22.222114 15318 raft_consensus.cc:2454] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e7c890486f846d495db1b41ab40a640 in term 1.
I20260811 16:09:22.223434 15015 leader_election.cc:302] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e7c890486f846d495db1b41ab40a640, a8620486dc7c4c92b88a816c9f29cdfd; no voters:
I20260811 16:09:22.224113 15412 raft_consensus.cc:2781] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 16:09:22.225028 15412 raft_consensus.cc:686] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 LEADER]: Becoming Leader. State: Replica: 3e7c890486f846d495db1b41ab40a640, State: Running, Role: LEADER
I20260811 16:09:22.225818 15412 consensus_queue.cc:237] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } }
I20260811 16:09:22.232306 14794 catalog_manager.cc:5495] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e7c890486f846d495db1b41ab40a640 (127.7.228.2). New cstate: current_term: 1 leader_uuid: "3e7c890486f846d495db1b41ab40a640" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 } health_report { overall_health: UNKNOWN } } }
I20260811 16:09:22.511039 14997 heartbeater.cc:503] Master 127.7.228.62:45977 was elected leader, sending a full tablet report...
I20260811 16:09:22.534109 15058 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:22.536473 14937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:22.537264 15303 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:22.540670 15177 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:22.704564 15412 consensus_queue.cc:1035] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 16:09:22.765092 15412 consensus_queue.cc:1035] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a924b46d9a864a67a9c3a8812500540a" member_type: VOTER last_known_addr { host: "127.7.228.3" port: 35927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
ba9d28588c2f42059fac1fd2d2eba91c | 127.7.228.62:45977 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
3e7c890486f846d495db1b41ab40a640 | 127.7.228.2:46297 | HEALTHY | <none> | 1 | 0
9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | HEALTHY | <none> | 0 | 0
a8620486dc7c4c92b88a816c9f29cdfd | 127.7.228.4:33275 | HEALTHY | <none> | 0 | 0
a924b46d9a864a67a9c3a8812500540a | 127.7.228.3:35927 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet c93c8c092e6c4941a662e44c99eaa583 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
a8620486dc7c4c92b88a816c9f29cdfd (127.7.228.4:33275): RUNNING
3e7c890486f846d495db1b41ab40a640 (127.7.228.2:46297): RUNNING [LEADER]
a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927): RUNNING
All reported replicas are:
A = a8620486dc7c4c92b88a816c9f29cdfd
B = 3e7c890486f846d495db1b41ab40a640
C = a924b46d9a864a67a9c3a8812500540a
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+-------------------+---------------
Small | 9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 16:09:22.825564 8080 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 16:09:23.215837 14937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:23.221103 15303 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:23.222373 15058 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:23.226775 15177 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
ba9d28588c2f42059fac1fd2d2eba91c | 127.7.228.62:45977 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
3e7c890486f846d495db1b41ab40a640 | 127.7.228.2:46297 | HEALTHY | <none> | 1 | 0
9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | HEALTHY | <none> | 0 | 0
a8620486dc7c4c92b88a816c9f29cdfd | 127.7.228.4:33275 | HEALTHY | <none> | 0 | 0
a924b46d9a864a67a9c3a8812500540a | 127.7.228.3:35927 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+-------------------+---------------
Small | 9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 16:09:23.451738 8080 log_verifier.cc:126] Checking tablet c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:23.513793 8080 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:23.514310 8080 tablet_replica.cc:330] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: stopping tablet replica
I20260811 16:09:23.514884 8080 raft_consensus.cc:2229] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:23.515336 8080 raft_consensus.cc:2258] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:23.758929 15058 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:23.764487 15303 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:23.769892 15177 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:23.770529 14937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:23.886109 15015 consensus_queue.cc:579] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: The tablet replica hosted on peer a924b46d9a864a67a9c3a8812500540a has failed
W20260811 16:09:23.886698 15015 consensus_peers.cc:487] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 -> Peer a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927): Couldn't send request to peer a924b46d9a864a67a9c3a8812500540a. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:23.898473 15075 consensus_queue.cc:237] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } }
I20260811 16:09:23.913568 15318 raft_consensus.cc:1261] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Refusing update from remote peer 3e7c890486f846d495db1b41ab40a640: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 16:09:23.915503 15594 consensus_queue.cc:1035] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 16:09:23.938280 15593 raft_consensus.cc:2932] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a924b46d9a864a67a9c3a8812500540a (127.7.228.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } }
I20260811 16:09:23.940332 15318 raft_consensus.cc:2932] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a924b46d9a864a67a9c3a8812500540a (127.7.228.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } }
I20260811 16:09:23.957834 14779 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c93c8c092e6c4941a662e44c99eaa583 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260811 16:09:23.964059 14793 catalog_manager.cc:5495] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 reported cstate change: config changed from index -1 to 2, VOTER a924b46d9a864a67a9c3a8812500540a (127.7.228.3) evicted. New cstate: current_term: 1 leader_uuid: "3e7c890486f846d495db1b41ab40a640" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } health_report { overall_health: HEALTHY } } }
I20260811 16:09:23.994069 15177 tablet_service.cc:1514] Processing DeleteTablet for tablet c93c8c092e6c4941a662e44c99eaa583 with delete_type TABLET_DATA_TOMBSTONED (TS a924b46d9a864a67a9c3a8812500540a not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:37284
I20260811 16:09:23.998687 15075 consensus_queue.cc:237] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } }
I20260811 16:09:24.002389 15615 ts_tablet_manager.cc:1905] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 16:09:24.013837 15318 raft_consensus.cc:1261] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Refusing update from remote peer 3e7c890486f846d495db1b41ab40a640: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260811 16:09:24.016382 15593 consensus_queue.cc:1035] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 16:09:24.019660 15615 ts_tablet_manager.cc:1918] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260811 16:09:24.020100 15615 log.cc:1196] T c93c8c092e6c4941a662e44c99eaa583 P a924b46d9a864a67a9c3a8812500540a: Deleting WAL directory at /tmp/dist-test-taskpZth0d/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786464498564520-8080-0/minicluster-data/ts-2-root/wals/c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:24.022056 14776 catalog_manager.cc:4841] TS a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927): tablet c93c8c092e6c4941a662e44c99eaa583 (table test-workload [id=2972837391624076ba8435f9cb8e10b3]) successfully deleted
I20260811 16:09:24.024547 15594 raft_consensus.cc:2932] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9d336df8967d4736aa84ee6e4d26e969 (127.7.228.1) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } } }
I20260811 16:09:24.026782 15318 raft_consensus.cc:2932] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9d336df8967d4736aa84ee6e4d26e969 (127.7.228.1) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } } }
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
ba9d28588c2f42059fac1fd2d2eba91c | 127.7.228.62:45977 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
3e7c890486f846d495db1b41ab40a640 | 127.7.228.2:46297 | HEALTHY | <none> | 1 | 0
9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | HEALTHY | <none> | 0 | 0
a8620486dc7c4c92b88a816c9f29cdfd | 127.7.228.4:33275 | HEALTHY | <none> | 0 | 0
a924b46d9a864a67a9c3a8812500540a | 127.7.228.3:35927 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet c93c8c092e6c4941a662e44c99eaa583 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
a8620486dc7c4c92b88a816c9f29cdfd (127.7.228.4:33275): RUNNING
3e7c890486f846d495db1b41ab40a640 (127.7.228.2:46297): RUNNING [LEADER]
a924b46d9a864a67a9c3a8812500540a (127.7.228.3:35927): not running
I20260811 16:09:24.034798 14779 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c93c8c092e6c4941a662e44c99eaa583 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = a8620486dc7c4c92b88a816c9f29cdfd
B = 3e7c890486f846d495db1b41ab40a640
C = a924b46d9a864a67a9c3a8812500540a
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+-------------------+---------------
Small | 9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 16:09:24.039616 14793 catalog_manager.cc:5495] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 reported cstate change: config changed from index 2 to 3, NON_VOTER 9d336df8967d4736aa84ee6e4d26e969 (127.7.228.1) added. New cstate: current_term: 1 leader_uuid: "3e7c890486f846d495db1b41ab40a640" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260811 16:09:24.042945 15013 consensus_peers.cc:487] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 -> Peer 9d336df8967d4736aa84ee6e4d26e969 (127.7.228.1:45239): Couldn't send request to peer 9d336df8967d4736aa84ee6e4d26e969. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c93c8c092e6c4941a662e44c99eaa583. This is attempt 1: this message will repeat every 5th retry.
I20260811 16:09:24.045172 8080 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 16:09:24.418748 15058 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260811 16:09:24.429735 14937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:24.436870 15303 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:24.441619 15177 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260811 16:09:24.481201 15657 ts_tablet_manager.cc:927] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Initiating tablet copy from peer 3e7c890486f846d495db1b41ab40a640 (127.7.228.2:46297)
I20260811 16:09:24.520727 15657 tablet_copy_client.cc:316] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: tablet copy: Beginning tablet copy session from remote peer at address 127.7.228.2:46297
I20260811 16:09:24.557564 15093 tablet_copy_service.cc:140] P 3e7c890486f846d495db1b41ab40a640: Received BeginTabletCopySession request for tablet c93c8c092e6c4941a662e44c99eaa583 from peer 9d336df8967d4736aa84ee6e4d26e969 ({username='slave'} at 127.0.0.1:42640)
I20260811 16:09:24.558182 15093 tablet_copy_service.cc:161] P 3e7c890486f846d495db1b41ab40a640: Beginning new tablet copy session on tablet c93c8c092e6c4941a662e44c99eaa583 from peer 9d336df8967d4736aa84ee6e4d26e969 at {username='slave'} at 127.0.0.1:42640: session id = 9d336df8967d4736aa84ee6e4d26e969-c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:24.566870 15093 tablet_copy_source_session.cc:205] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 16:09:24.570600 15657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93c8c092e6c4941a662e44c99eaa583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 16:09:24.590318 15657 tablet_copy_client.cc:791] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: tablet copy: Starting download of 0 data blocks...
I20260811 16:09:24.591027 15657 tablet_copy_client.cc:655] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: tablet copy: Starting download of 1 WAL segments...
I20260811 16:09:24.595299 15657 tablet_copy_client.cc:531] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 16:09:24.605283 15657 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Bootstrap starting.
I20260811 16:09:24.639321 15657 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 16:09:24.641072 15657 tablet_bootstrap.cc:492] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Bootstrap complete.
I20260811 16:09:24.641834 15657 ts_tablet_manager.cc:1397] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Time spent bootstrapping tablet: real 0.037s user 0.014s sys 0.015s
I20260811 16:09:24.644996 15657 raft_consensus.cc:348] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } }
I20260811 16:09:24.645840 15657 raft_consensus.cc:729] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9d336df8967d4736aa84ee6e4d26e969, State: Initialized, Role: LEARNER
I20260811 16:09:24.646485 15657 consensus_queue.cc:260] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a8620486dc7c4c92b88a816c9f29cdfd" member_type: VOTER last_known_addr { host: "127.7.228.4" port: 33275 } } peers { permanent_uuid: "3e7c890486f846d495db1b41ab40a640" member_type: VOTER last_known_addr { host: "127.7.228.2" port: 46297 } } peers { permanent_uuid: "9d336df8967d4736aa84ee6e4d26e969" member_type: NON_VOTER last_known_addr { host: "127.7.228.1" port: 45239 } attrs { promote: true } }
I20260811 16:09:24.662542 15657 ts_tablet_manager.cc:1428] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: Time spent starting tablet: real 0.020s user 0.002s sys 0.018s
I20260811 16:09:24.664325 15093 tablet_copy_service.cc:342] P 3e7c890486f846d495db1b41ab40a640: Request end of tablet copy session 9d336df8967d4736aa84ee6e4d26e969-c93c8c092e6c4941a662e44c99eaa583 received from {username='slave'} at 127.0.0.1:42640
I20260811 16:09:24.664757 15093 tablet_copy_service.cc:433] P 3e7c890486f846d495db1b41ab40a640: ending tablet copy session 9d336df8967d4736aa84ee6e4d26e969-c93c8c092e6c4941a662e44c99eaa583 on tablet c93c8c092e6c4941a662e44c99eaa583 with peer 9d336df8967d4736aa84ee6e4d26e969
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
ba9d28588c2f42059fac1fd2d2eba91c | 127.7.228.62:45977 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
3e7c890486f846d495db1b41ab40a640 | 127.7.228.2:46297 | HEALTHY | <none> | 1 | 0
9d336df8967d4736aa84ee6e4d26e969 | 127.7.228.1:45239 | HEALTHY | <none> | 0 | 0
a8620486dc7c4c92b88a816c9f29cdfd | 127.7.228.4:33275 | HEALTHY | <none> | 0 | 0
a924b46d9a864a67a9c3a8812500540a | 127.7.228.3:35927 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+-------------------+---------------
Small | a924b46d9a864a67a9c3a8812500540a | 127.7.228.3:35927 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 16:09:24.718470 8080 log_verifier.cc:126] Checking tablet c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:24.776407 8080 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c93c8c092e6c4941a662e44c99eaa583
I20260811 16:09:24.778470 8080 tablet_server.cc:178] TabletServer@127.7.228.1:0 shutting down...
I20260811 16:09:24.800484 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:24.801491 8080 tablet_replica.cc:330] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969: stopping tablet replica
I20260811 16:09:24.802076 8080 raft_consensus.cc:2229] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 16:09:24.802662 8080 raft_consensus.cc:2258] T c93c8c092e6c4941a662e44c99eaa583 P 9d336df8967d4736aa84ee6e4d26e969 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 16:09:24.823098 8080 tablet_server.cc:194] TabletServer@127.7.228.1:0 shutdown complete.
I20260811 16:09:24.833176 8080 tablet_server.cc:178] TabletServer@127.7.228.2:0 shutting down...
I20260811 16:09:24.852699 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:24.853349 8080 tablet_replica.cc:330] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640: stopping tablet replica
I20260811 16:09:24.853950 8080 raft_consensus.cc:2229] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:24.854831 8080 raft_consensus.cc:2258] T c93c8c092e6c4941a662e44c99eaa583 P 3e7c890486f846d495db1b41ab40a640 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:24.875034 8080 tablet_server.cc:194] TabletServer@127.7.228.2:0 shutdown complete.
I20260811 16:09:24.885143 8080 tablet_server.cc:178] TabletServer@127.7.228.3:0 shutting down...
I20260811 16:09:24.906136 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:24.923538 8080 tablet_server.cc:194] TabletServer@127.7.228.3:0 shutdown complete.
I20260811 16:09:24.933094 8080 tablet_server.cc:178] TabletServer@127.7.228.4:0 shutting down...
I20260811 16:09:24.952524 8080 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260811 16:09:24.953140 8080 tablet_replica.cc:330] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd: stopping tablet replica
I20260811 16:09:24.953672 8080 raft_consensus.cc:2229] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 16:09:24.954097 8080 raft_consensus.cc:2258] T c93c8c092e6c4941a662e44c99eaa583 P a8620486dc7c4c92b88a816c9f29cdfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:24.973942 8080 tablet_server.cc:194] TabletServer@127.7.228.4:0 shutdown complete.
I20260811 16:09:24.983781 8080 master.cc:537] Master@127.7.228.62:45977 shutting down...
I20260811 16:09:25.000675 8080 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 1 LEADER]: Raft consensus shutting down.
I20260811 16:09:25.001245 8080 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 16:09:25.001540 8080 tablet_replica.cc:330] T 00000000000000000000000000000000 P ba9d28588c2f42059fac1fd2d2eba91c: stopping tablet replica
I20260811 16:09:25.019412 8080 master.cc:552] Master@127.7.228.62:45977 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (3893 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (8195 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (66312 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260811 16:09:25.054368 8080 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~9 seconds ago
I20260811 16:09:25.054705 8080 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 98 messages since previous log ~57 seconds ago
I20260811 16:09:25.054879 8080 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 14 messages since previous log ~14 seconds ago
I20260811 16:09:25.055176 8080 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 8 messages since previous log ~12 seconds ago