[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:54:10.612553 5300 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.226s user 0.393s sys 0.831s
W20260807 15:54:10.613152 5300 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.227s user 0.393s sys 0.831s
I20260807 15:54:10.615968 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:37737
I20260807 15:54:10.617954 5300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:54:10.619266 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:10.637089 5313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:10.637116 5314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:10.638126 5316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:10.738960 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:10.740278 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:10.740533 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:10.740732 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118050740708 us; error 0 us; skew 500 ppm
I20260807 15:54:10.741580 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:10.746865 5300 webserver.cc:460] Webserver started at http://127.5.45.62:33379/ using document root <none> and password file <none>
I20260807 15:54:10.747709 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:10.747942 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:10.748378 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:10.754727 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "abd210d293da43b184651a09b47259f3"
format_stamp: "Formatted at 2026-08-07 15:54:10 on dist-test-slave-bw88"
I20260807 15:54:10.762763 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 15:54:10.768002 5323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:10.769383 5300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 15:54:10.769761 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "abd210d293da43b184651a09b47259f3"
format_stamp: "Formatted at 2026-08-07 15:54:10 on dist-test-slave-bw88"
I20260807 15:54:10.770115 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:10.793610 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:10.795315 5300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:54:10.795825 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:10.870270 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:37737
I20260807 15:54:10.870290 5389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:37737 every 8 connection(s)
I20260807 15:54:10.876789 5390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:10.901199 5390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: Bootstrap starting.
I20260807 15:54:10.907277 5390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:10.909118 5390 log.cc:824] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:54:10.913841 5390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: No bootstrap required, opened a new log
I20260807 15:54:10.936043 5390 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER }
I20260807 15:54:10.936704 5390 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:10.937048 5390 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd210d293da43b184651a09b47259f3, State: Initialized, Role: FOLLOWER
I20260807 15:54:10.937992 5390 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER }
I20260807 15:54:10.938524 5390 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:54:10.938838 5390 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:54:10.939172 5390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:10.943369 5390 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER }
I20260807 15:54:10.944123 5390 leader_election.cc:302] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd210d293da43b184651a09b47259f3; no voters:
I20260807 15:54:10.945977 5390 leader_election.cc:288] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:54:10.946278 5394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:10.949152 5394 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 1 LEADER]: Becoming Leader. State: Replica: abd210d293da43b184651a09b47259f3, State: Running, Role: LEADER
I20260807 15:54:10.949982 5394 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER }
I20260807 15:54:10.951043 5390 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:10.959853 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd210d293da43b184651a09b47259f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER } }
I20260807 15:54:10.958921 5397 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd210d293da43b184651a09b47259f3. Latest consensus state: current_term: 1 leader_uuid: "abd210d293da43b184651a09b47259f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd210d293da43b184651a09b47259f3" member_type: VOTER } }
I20260807 15:54:10.960702 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:10.961022 5397 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:10.965265 5404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:10.977176 5404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:10.979084 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:10.993953 5404 catalog_manager.cc:1347] Generated new cluster ID: 8656d17b6dd2404da41e9ba0048038d4
I20260807 15:54:10.994304 5404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:11.012130 5404 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:11.014972 5404 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:11.047953 5404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: Generated new TSK 0
I20260807 15:54:11.049010 5404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:11.110725 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:11.119940 5415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:11.121076 5416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:11.126099 5419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:11.127056 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:11.128010 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:11.128298 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:11.128502 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118051128484 us; error 0 us; skew 500 ppm
I20260807 15:54:11.129299 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:11.133724 5300 webserver.cc:460] Webserver started at http://127.5.45.1:46619/ using document root <none> and password file <none>
I20260807 15:54:11.134475 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:11.134737 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:11.135102 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:11.136897 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "2344ae131c714fa8986d4e1e1832a29d"
format_stamp: "Formatted at 2026-08-07 15:54:11 on dist-test-slave-bw88"
I20260807 15:54:11.144407 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 15:54:11.150018 5426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.150983 5300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 15:54:11.151347 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "2344ae131c714fa8986d4e1e1832a29d"
format_stamp: "Formatted at 2026-08-07 15:54:11 on dist-test-slave-bw88"
I20260807 15:54:11.151762 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:11.233031 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:11.234498 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:11.236935 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:11.241835 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:11.242125 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.242413 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:11.242602 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.358525 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:43309
I20260807 15:54:11.359149 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:43309 every 8 connection(s)
I20260807 15:54:11.366014 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:11.379262 5509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:11.383502 5511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:11.392922 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:11.393261 5513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:11.394344 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:11.394589 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:11.394776 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118051394759 us; error 0 us; skew 500 ppm
I20260807 15:54:11.395431 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:11.399921 5300 webserver.cc:460] Webserver started at http://127.5.45.2:43703/ using document root <none> and password file <none>
I20260807 15:54:11.400550 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:11.400800 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:11.401119 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:11.401517 5504 heartbeater.cc:347] Connected to a master server at 127.5.45.62:37737
I20260807 15:54:11.401980 5504 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:11.402469 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "4ec2763d2c914a15852f6937adb9a30d"
format_stamp: "Formatted at 2026-08-07 15:54:11 on dist-test-slave-bw88"
I20260807 15:54:11.403048 5504 heartbeater.cc:511] Master 127.5.45.62:37737 requested a full tablet report, sending...
I20260807 15:54:11.406579 5344 ts_manager.cc:194] Registered new tserver with Master: 2344ae131c714fa8986d4e1e1832a29d (127.5.45.1:43309)
I20260807 15:54:11.409119 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 15:54:11.410009 5344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58726
I20260807 15:54:11.413803 5519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.414706 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:54:11.414984 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "4ec2763d2c914a15852f6937adb9a30d"
format_stamp: "Formatted at 2026-08-07 15:54:11 on dist-test-slave-bw88"
I20260807 15:54:11.415282 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:11.443225 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:11.444527 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:11.447324 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:11.450377 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:11.450636 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.450908 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:11.451100 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:11.514706 5504 heartbeater.cc:503] Master 127.5.45.62:37737 was elected leader, sending a full tablet report...
I20260807 15:54:11.550336 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:44853
I20260807 15:54:11.550522 5605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:44853 every 8 connection(s)
I20260807 15:54:11.569437 5606 heartbeater.cc:347] Connected to a master server at 127.5.45.62:37737
I20260807 15:54:11.569814 5606 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:11.570572 5606 heartbeater.cc:511] Master 127.5.45.62:37737 requested a full tablet report, sending...
I20260807 15:54:11.572674 5344 ts_manager.cc:194] Registered new tserver with Master: 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2:44853)
I20260807 15:54:11.572945 5300 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.01802768s
I20260807 15:54:11.574836 5344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58740
I20260807 15:54:11.609848 5344 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:54:11.633013 5344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260807 15:54:11.678596 5606 heartbeater.cc:503] Master 127.5.45.62:37737 was elected leader, sending a full tablet report...
I20260807 15:54:11.702653 5559 tablet_service.cc:1467] Processing CreateTablet for tablet 398628cd2ab24e0193ff7e1bce0be741 (DEFAULT_TABLE table=test-table [id=14aee77d34af4c169d41ed137b9adc3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:11.703349 5561 tablet_service.cc:1467] Processing CreateTablet for tablet 18f495b038804e59a78b5758af110f17 (DEFAULT_TABLE table=test-table [id=14aee77d34af4c169d41ed137b9adc3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:11.704470 5559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398628cd2ab24e0193ff7e1bce0be741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:11.705052 5561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f495b038804e59a78b5758af110f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:11.715056 5458 tablet_service.cc:1467] Processing CreateTablet for tablet 18f495b038804e59a78b5758af110f17 (DEFAULT_TABLE table=test-table [id=14aee77d34af4c169d41ed137b9adc3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:11.716478 5458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f495b038804e59a78b5758af110f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:11.722427 5457 tablet_service.cc:1467] Processing CreateTablet for tablet 398628cd2ab24e0193ff7e1bce0be741 (DEFAULT_TABLE table=test-table [id=14aee77d34af4c169d41ed137b9adc3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:11.723506 5457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398628cd2ab24e0193ff7e1bce0be741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:11.730829 5628 tablet_bootstrap.cc:492] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: Bootstrap starting.
I20260807 15:54:11.737668 5628 tablet_bootstrap.cc:654] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:11.746690 5629 tablet_bootstrap.cc:492] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: Bootstrap starting.
I20260807 15:54:11.751809 5628 tablet_bootstrap.cc:492] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: No bootstrap required, opened a new log
I20260807 15:54:11.752310 5628 ts_tablet_manager.cc:1397] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.015s
I20260807 15:54:11.753904 5629 tablet_bootstrap.cc:654] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:11.755285 5628 raft_consensus.cc:348] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:11.756080 5628 raft_consensus.cc:729] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Initialized, Role: FOLLOWER
I20260807 15:54:11.756731 5628 consensus_queue.cc:260] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:11.761102 5628 ts_tablet_manager.cc:1428] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: Time spent starting tablet: real 0.008s user 0.010s sys 0.000s
I20260807 15:54:11.761772 5629 tablet_bootstrap.cc:492] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: No bootstrap required, opened a new log
I20260807 15:54:11.762224 5629 ts_tablet_manager.cc:1397] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260807 15:54:11.763118 5628 tablet_bootstrap.cc:492] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: Bootstrap starting.
I20260807 15:54:11.764799 5629 raft_consensus.cc:348] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:11.765523 5629 raft_consensus.cc:729] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2344ae131c714fa8986d4e1e1832a29d, State: Initialized, Role: FOLLOWER
I20260807 15:54:11.766103 5629 consensus_queue.cc:260] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:11.771422 5628 tablet_bootstrap.cc:654] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:11.777415 5629 ts_tablet_manager.cc:1428] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260807 15:54:11.781626 5629 tablet_bootstrap.cc:492] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: Bootstrap starting.
I20260807 15:54:11.787786 5629 tablet_bootstrap.cc:654] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:11.792440 5628 tablet_bootstrap.cc:492] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: No bootstrap required, opened a new log
I20260807 15:54:11.792891 5628 ts_tablet_manager.cc:1397] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.015s
I20260807 15:54:11.795512 5628 raft_consensus.cc:348] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:11.796178 5628 raft_consensus.cc:729] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Initialized, Role: FOLLOWER
I20260807 15:54:11.796747 5628 consensus_queue.cc:260] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:11.799016 5628 ts_tablet_manager.cc:1428] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260807 15:54:11.807070 5607 tablet.cc:2369] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:54:11.810144 5629 tablet_bootstrap.cc:492] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: No bootstrap required, opened a new log
I20260807 15:54:11.810809 5629 ts_tablet_manager.cc:1397] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: Time spent bootstrapping tablet: real 0.029s user 0.017s sys 0.006s
I20260807 15:54:11.813455 5629 raft_consensus.cc:348] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:11.813954 5629 raft_consensus.cc:729] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2344ae131c714fa8986d4e1e1832a29d, State: Initialized, Role: FOLLOWER
I20260807 15:54:11.814492 5629 consensus_queue.cc:260] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:11.816767 5629 ts_tablet_manager.cc:1428] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 15:54:11.830444 5300 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260807 15:54:11.830760 5300 raft_consensus.cc:412] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Emulating election...
I20260807 15:54:11.830997 5300 raft_consensus.cc:3037] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:11.834853 5300 raft_consensus.cc:686] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Becoming Leader. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:11.835523 5300 consensus_queue.cc:237] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:11.838802 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:11.845486 5342 catalog_manager.cc:5495] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2). New cstate: current_term: 1 leader_uuid: "4ec2763d2c914a15852f6937adb9a30d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:11.868376 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:11.972435 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:12.176386 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:12.405251 5473 raft_consensus.cc:3037] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:12.411849 5641 consensus_queue.cc:1035] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:12.481001 5300 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 15:54:12.481362 5300 raft_consensus.cc:412] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Emulating election...
I20260807 15:54:12.481662 5300 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:12.487808 5300 raft_consensus.cc:686] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Becoming Leader. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:12.488505 5300 consensus_queue.cc:237] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:12.491122 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:12.494988 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:12.495265 5342 catalog_manager.cc:5495] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2). New cstate: current_term: 1 leader_uuid: "4ec2763d2c914a15852f6937adb9a30d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:12.599004 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:12.803005 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260807 15:54:12.891247 5473 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:12.897300 5641 consensus_queue.cc:1035] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:13.307338 5300 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 15:54:13.307581 5300 raft_consensus.cc:412] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Emulating election...
I20260807 15:54:13.307859 5300 raft_consensus.cc:3032] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:13.308045 5300 raft_consensus.cc:729] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:13.308561 5300 consensus_queue.cc:260] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:13.309448 5300 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:13.313445 5300 raft_consensus.cc:686] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 2 LEADER]: Becoming Leader. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:13.314066 5300 consensus_queue.cc:237] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:13.315699 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:13.319729 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 15:54:13.319366 5343 catalog_manager.cc:5495] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "4ec2763d2c914a15852f6937adb9a30d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:13.423959 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 15:54:13.627686 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260807 15:54:13.842419 5473 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:13.848419 5473 raft_consensus.cc:1261] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 2 FOLLOWER]: Refusing update from remote peer 4ec2763d2c914a15852f6937adb9a30d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:54:13.849766 5648 consensus_queue.cc:1035] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:13.931303 5300 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 15:54:13.931582 5300 raft_consensus.cc:412] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 2 FOLLOWER]: Emulating election...
I20260807 15:54:13.931818 5300 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:13.935681 5300 raft_consensus.cc:686] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 3 LEADER]: Becoming Leader. State: Replica: 2344ae131c714fa8986d4e1e1832a29d, State: Running, Role: LEADER
I20260807 15:54:13.936324 5300 consensus_queue.cc:237] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:13.938094 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:13.941798 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 15:54:13.941593 5342 catalog_manager.cc:5495] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d reported cstate change: term changed from 2 to 3, leader changed from 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2) to 2344ae131c714fa8986d4e1e1832a29d (127.5.45.1). New cstate: current_term: 3 leader_uuid: "2344ae131c714fa8986d4e1e1832a29d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:14.045571 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 15:54:14.250278 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 15:54:14.280501 5473 raft_consensus.cc:1226] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 3 LEADER]: Rejecting Update request from peer 4ec2763d2c914a15852f6937adb9a30d for earlier term 2. Current term is 3. Ops: []
I20260807 15:54:14.281764 5648 consensus_queue.cc:1046] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 15:54:14.282462 5648 consensus_queue.cc:1225] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Peer 2344ae131c714fa8986d4e1e1832a29d log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 15:54:14.282869 5662 raft_consensus.cc:3032] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 2 LEADER]: Stepping down as leader of term 2
I20260807 15:54:14.283115 5662 raft_consensus.cc:729] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:14.283560 5662 consensus_queue.cc:260] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:14.284260 5662 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:14.554713 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260807 15:54:14.574787 5574 raft_consensus.cc:1261] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 3 FOLLOWER]: Refusing update from remote peer 2344ae131c714fa8986d4e1e1832a29d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 15:54:14.576277 5668 consensus_queue.cc:1035] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 15:54:15.158870 5300 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260807 15:54:15.159212 5300 raft_consensus.cc:412] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 1 FOLLOWER]: Emulating election...
I20260807 15:54:15.159472 5300 raft_consensus.cc:3037] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:15.164141 5300 raft_consensus.cc:686] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 2 LEADER]: Becoming Leader. State: Replica: 2344ae131c714fa8986d4e1e1832a29d, State: Running, Role: LEADER
I20260807 15:54:15.164793 5300 consensus_queue.cc:237] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:15.166800 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:15.170552 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 15:54:15.170214 5343 catalog_manager.cc:5495] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d reported cstate change: term changed from 1 to 2, leader changed from 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2) to 2344ae131c714fa8986d4e1e1832a29d (127.5.45.1). New cstate: current_term: 2 leader_uuid: "2344ae131c714fa8986d4e1e1832a29d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:15.274430 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 15:54:15.469774 5473 raft_consensus.cc:1226] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 2 LEADER]: Rejecting Update request from peer 4ec2763d2c914a15852f6937adb9a30d for earlier term 1. Current term is 2. Ops: []
I20260807 15:54:15.471045 5662 consensus_queue.cc:1046] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:15.472765 5662 consensus_queue.cc:1225] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Peer 2344ae131c714fa8986d4e1e1832a29d log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 15:54:15.473237 5684 raft_consensus.cc:3032] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:15.473549 5684 raft_consensus.cc:729] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:15.474063 5684 consensus_queue.cc:260] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } } peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } }
I20260807 15:54:15.474812 5684 raft_consensus.cc:3037] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:15.482007 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260807 15:54:15.624903 5574 raft_consensus.cc:1261] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 2 FOLLOWER]: Refusing update from remote peer 2344ae131c714fa8986d4e1e1832a29d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:54:15.626528 5676 consensus_queue.cc:1035] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:15.785737 5300 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260807 15:54:15.785989 5300 raft_consensus.cc:412] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 3 FOLLOWER]: Emulating election...
I20260807 15:54:15.786237 5300 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:15.790308 5300 raft_consensus.cc:686] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 4 LEADER]: Becoming Leader. State: Replica: 4ec2763d2c914a15852f6937adb9a30d, State: Running, Role: LEADER
I20260807 15:54:15.790897 5300 consensus_queue.cc:237] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:15.792835 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:15.796684 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 15:54:15.796689 5344 catalog_manager.cc:5495] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d reported cstate change: term changed from 3 to 4, leader changed from 2344ae131c714fa8986d4e1e1832a29d (127.5.45.1) to 4ec2763d2c914a15852f6937adb9a30d (127.5.45.2). New cstate: current_term: 4 leader_uuid: "4ec2763d2c914a15852f6937adb9a30d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:15.900645 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 15:54:16.015307 5574 raft_consensus.cc:1226] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 4 LEADER]: Rejecting Update request from peer 2344ae131c714fa8986d4e1e1832a29d for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:16.016530 5676 consensus_queue.cc:1046] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 15:54:16.017220 5676 consensus_queue.cc:1225] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [LEADER]: Peer 4ec2763d2c914a15852f6937adb9a30d log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 15:54:16.017655 5686 raft_consensus.cc:3032] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 3 LEADER]: Stepping down as leader of term 3
I20260807 15:54:16.017925 5686 raft_consensus.cc:729] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2344ae131c714fa8986d4e1e1832a29d, State: Running, Role: LEADER
I20260807 15:54:16.018451 5686 consensus_queue.cc:260] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 } } peers { permanent_uuid: "4ec2763d2c914a15852f6937adb9a30d" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 44853 } }
I20260807 15:54:16.019169 5686 raft_consensus.cc:3037] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:16.104913 5300 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260807 15:54:16.197789 5473 raft_consensus.cc:1261] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 4 FOLLOWER]: Refusing update from remote peer 4ec2763d2c914a15852f6937adb9a30d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 15:54:16.199116 5684 consensus_queue.cc:1035] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344ae131c714fa8986d4e1e1832a29d" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 15:54:16.611307 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:54:16.641093 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:16.641812 5300 tablet_replica.cc:330] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d: stopping tablet replica
I20260807 15:54:16.642727 5300 raft_consensus.cc:2229] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 15:54:16.643210 5300 raft_consensus.cc:2258] T 398628cd2ab24e0193ff7e1bce0be741 P 2344ae131c714fa8986d4e1e1832a29d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:16.645920 5300 tablet_replica.cc:330] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d: stopping tablet replica
I20260807 15:54:16.646317 5300 raft_consensus.cc:2229] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 2 LEADER]: Raft consensus shutting down.
I20260807 15:54:16.646837 5300 raft_consensus.cc:2258] T 18f495b038804e59a78b5758af110f17 P 2344ae131c714fa8986d4e1e1832a29d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:16.743054 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:54:16.752923 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:54:16.776156 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:16.776734 5300 tablet_replica.cc:330] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d: stopping tablet replica
I20260807 15:54:16.777222 5300 raft_consensus.cc:2229] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 15:54:16.777697 5300 raft_consensus.cc:2258] T 18f495b038804e59a78b5758af110f17 P 4ec2763d2c914a15852f6937adb9a30d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:16.779098 5300 tablet_replica.cc:330] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d: stopping tablet replica
I20260807 15:54:16.779506 5300 raft_consensus.cc:2229] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 4 LEADER]: Raft consensus shutting down.
I20260807 15:54:16.780062 5300 raft_consensus.cc:2258] T 398628cd2ab24e0193ff7e1bce0be741 P 4ec2763d2c914a15852f6937adb9a30d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:16.799259 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:54:16.807332 5300 master.cc:537] Master@127.5.45.62:37737 shutting down...
I20260807 15:54:17.010440 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:17.011040 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:17.011435 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P abd210d293da43b184651a09b47259f3: stopping tablet replica
I20260807 15:54:17.028385 5300 master.cc:552] Master@127.5.45.62:37737 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7702 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260807 15:54:17.059087 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:41317
I20260807 15:54:17.060127 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.065419 5707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.065984 5710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.067687 5715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.068459 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:17.069087 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.069264 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.069612 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057069593 us; error 0 us; skew 500 ppm
I20260807 15:54:17.070116 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.078084 5300 webserver.cc:460] Webserver started at http://127.5.45.62:39719/ using document root <none> and password file <none>
I20260807 15:54:17.078594 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.078810 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.079077 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.080142 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "5f93b068878a4f6fb034b1a3301790f2"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.084518 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:54:17.090081 5723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.091079 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 15:54:17.091425 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "5f93b068878a4f6fb034b1a3301790f2"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.091791 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:17.104501 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:17.105569 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:17.142650 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:41317
I20260807 15:54:17.142750 5791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:41317 every 8 connection(s)
I20260807 15:54:17.146432 5792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:17.155545 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2: Bootstrap starting.
I20260807 15:54:17.159555 5792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:17.163457 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2: No bootstrap required, opened a new log
I20260807 15:54:17.165192 5792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER }
I20260807 15:54:17.165589 5792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:17.165761 5792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f93b068878a4f6fb034b1a3301790f2, State: Initialized, Role: FOLLOWER
I20260807 15:54:17.166213 5792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER }
I20260807 15:54:17.166608 5792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:54:17.166785 5792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:54:17.166980 5792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:17.170863 5792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER }
I20260807 15:54:17.171350 5792 leader_election.cc:302] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f93b068878a4f6fb034b1a3301790f2; no voters:
I20260807 15:54:17.172367 5792 leader_election.cc:288] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:54:17.172855 5796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:17.174257 5796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5f93b068878a4f6fb034b1a3301790f2, State: Running, Role: LEADER
I20260807 15:54:17.174911 5796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER }
I20260807 15:54:17.175223 5792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:17.178202 5798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f93b068878a4f6fb034b1a3301790f2. Latest consensus state: current_term: 1 leader_uuid: "5f93b068878a4f6fb034b1a3301790f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER } }
I20260807 15:54:17.178551 5797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f93b068878a4f6fb034b1a3301790f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f93b068878a4f6fb034b1a3301790f2" member_type: VOTER } }
I20260807 15:54:17.178866 5798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:17.179112 5797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:17.182502 5803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:17.187336 5803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:17.190552 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:17.195574 5803 catalog_manager.cc:1347] Generated new cluster ID: c20340b735d249fca9778d3078040651
I20260807 15:54:17.195923 5803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:17.230074 5803 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:17.231729 5803 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:17.245666 5803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2: Generated new TSK 0
I20260807 15:54:17.246390 5803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:17.257148 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.263751 5822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.264608 5824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.265930 5826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.266547 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:17.267343 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.267514 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.267629 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057267621 us; error 0 us; skew 500 ppm
I20260807 15:54:17.268052 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.270434 5300 webserver.cc:460] Webserver started at http://127.5.45.1:42055/ using document root <none> and password file <none>
I20260807 15:54:17.270869 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.271021 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.271220 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.272222 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "6c10d18b9c5941bd9c857f2392da6ecd"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.276773 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:54:17.279861 5832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.280632 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:54:17.280911 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "6c10d18b9c5941bd9c857f2392da6ecd"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.281214 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:17.292096 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:17.293421 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:17.294867 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:17.297065 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:17.297282 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.297595 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:17.297770 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.335388 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:39149
I20260807 15:54:17.335496 5918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:39149 every 8 connection(s)
I20260807 15:54:17.339802 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.346976 5926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.347919 5927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.349970 5930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.350618 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:17.351477 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.351708 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.351881 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057351870 us; error 0 us; skew 500 ppm
I20260807 15:54:17.352531 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.355381 5920 heartbeater.cc:347] Connected to a master server at 127.5.45.62:41317
I20260807 15:54:17.355798 5920 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:17.355757 5300 webserver.cc:460] Webserver started at http://127.5.45.2:35325/ using document root <none> and password file <none>
I20260807 15:54:17.356525 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.356730 5920 heartbeater.cc:511] Master 127.5.45.62:41317 requested a full tablet report, sending...
I20260807 15:54:17.356820 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.357262 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.358675 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "d901bc6d2c8e44ae9b941bfe2325c0fe"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.359129 5746 ts_manager.cc:194] Registered new tserver with Master: 6c10d18b9c5941bd9c857f2392da6ecd (127.5.45.1:39149)
I20260807 15:54:17.362015 5746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49018
I20260807 15:54:17.364403 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 15:54:17.367970 5936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.368764 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:54:17.369071 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "d901bc6d2c8e44ae9b941bfe2325c0fe"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.369403 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:17.388836 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:17.390012 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:17.391418 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:17.393846 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:17.394063 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.394299 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:17.394456 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.431839 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:45615
I20260807 15:54:17.431954 6016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:45615 every 8 connection(s)
I20260807 15:54:17.446879 6017 heartbeater.cc:347] Connected to a master server at 127.5.45.62:41317
I20260807 15:54:17.447263 6017 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:17.447906 6017 heartbeater.cc:511] Master 127.5.45.62:41317 requested a full tablet report, sending...
I20260807 15:54:17.449817 5746 ts_manager.cc:194] Registered new tserver with Master: d901bc6d2c8e44ae9b941bfe2325c0fe (127.5.45.2:45615)
I20260807 15:54:17.450667 5300 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015809759s
I20260807 15:54:17.451753 5746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49032
I20260807 15:54:17.465615 5920 heartbeater.cc:503] Master 127.5.45.62:41317 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260807 15:54:17.475078 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:54:17.492727 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:17.508764 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:54:17.515704 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:54:17.531980 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:17.547784 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:54:17.554849 5300 master.cc:537] Master@127.5.45.62:41317 shutting down...
I20260807 15:54:17.569090 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:17.569655 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:17.570032 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5f93b068878a4f6fb034b1a3301790f2: stopping tablet replica
I20260807 15:54:17.588977 5300 master.cc:552] Master@127.5.45.62:41317 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (551 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260807 15:54:17.610558 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:39023
I20260807 15:54:17.611579 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.616535 6032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.618429 6035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.619818 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:17.620285 6037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.621052 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.621307 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.621598 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057621579 us; error 0 us; skew 500 ppm
I20260807 15:54:17.622120 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.624680 5300 webserver.cc:460] Webserver started at http://127.5.45.62:45499/ using document root <none> and password file <none>
I20260807 15:54:17.625185 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.625412 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.625690 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.626789 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "f27e21e80bef41a299393e375075e630"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.631282 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 15:54:17.634622 6042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.635411 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260807 15:54:17.635708 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "f27e21e80bef41a299393e375075e630"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.636005 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:17.659699 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:17.660758 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:17.696949 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:39023
I20260807 15:54:17.697045 6108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:39023 every 8 connection(s)
I20260807 15:54:17.700631 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:17.710114 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630: Bootstrap starting.
I20260807 15:54:17.714223 6109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:17.718649 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630: No bootstrap required, opened a new log
I20260807 15:54:17.720466 6109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER }
I20260807 15:54:17.720875 6109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:17.721117 6109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27e21e80bef41a299393e375075e630, State: Initialized, Role: FOLLOWER
I20260807 15:54:17.721740 6109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER }
I20260807 15:54:17.722198 6109 raft_consensus.cc:388] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:54:17.722443 6109 raft_consensus.cc:482] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:54:17.722728 6109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:17.726852 6109 raft_consensus.cc:504] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER }
I20260807 15:54:17.727444 6109 leader_election.cc:302] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f27e21e80bef41a299393e375075e630; no voters:
I20260807 15:54:17.728578 6109 leader_election.cc:288] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:54:17.728897 6112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:17.730392 6112 raft_consensus.cc:686] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 1 LEADER]: Becoming Leader. State: Replica: f27e21e80bef41a299393e375075e630, State: Running, Role: LEADER
I20260807 15:54:17.731072 6112 consensus_queue.cc:237] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER }
I20260807 15:54:17.731527 6109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:17.733764 6115 sys_catalog.cc:455] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f27e21e80bef41a299393e375075e630. Latest consensus state: current_term: 1 leader_uuid: "f27e21e80bef41a299393e375075e630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER } }
I20260807 15:54:17.733763 6113 sys_catalog.cc:455] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f27e21e80bef41a299393e375075e630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f27e21e80bef41a299393e375075e630" member_type: VOTER } }
I20260807 15:54:17.734390 6115 sys_catalog.cc:458] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:17.734438 6113 sys_catalog.cc:458] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:17.737169 6119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:17.741983 6119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:17.746213 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:17.750124 6119 catalog_manager.cc:1347] Generated new cluster ID: 6fd9e69c5fff4715b9b0ba28d4253034
I20260807 15:54:17.750401 6119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:17.768746 6119 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:17.770371 6119 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:17.786803 6119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630: Generated new TSK 0
I20260807 15:54:17.787489 6119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:17.813669 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.819471 6135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.820668 6136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.821843 6138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.822593 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:17.823328 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.823537 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.823683 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057823672 us; error 0 us; skew 500 ppm
I20260807 15:54:17.824230 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.826689 5300 webserver.cc:460] Webserver started at http://127.5.45.1:37719/ using document root <none> and password file <none>
I20260807 15:54:17.827255 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.827467 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.827730 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.828806 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "84e939a6df4f4f148d10263f4f7e0a15"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.833534 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 15:54:17.836942 6145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.837831 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:54:17.838112 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "84e939a6df4f4f148d10263f4f7e0a15"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.838377 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:17.867182 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:17.868278 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:17.869736 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:17.872582 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:17.872759 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.873056 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:17.873208 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.938791 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:33435
I20260807 15:54:17.938895 6230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:33435 every 8 connection(s)
I20260807 15:54:17.944209 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:17.951406 6235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.953704 6236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:17.956099 6238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:17.959578 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:17.960413 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:17.960592 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:17.960716 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118057960708 us; error 0 us; skew 500 ppm
I20260807 15:54:17.961326 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:17.964506 5300 webserver.cc:460] Webserver started at http://127.5.45.2:46349/ using document root <none> and password file <none>
I20260807 15:54:17.964956 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:17.965154 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:17.965512 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:17.966795 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "0976de1ffe0f4c66b303bbf70baf3098"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.967648 6231 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39023
I20260807 15:54:17.968034 6231 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:17.968672 6231 heartbeater.cc:511] Master 127.5.45.62:39023 requested a full tablet report, sending...
I20260807 15:54:17.970929 6067 ts_manager.cc:194] Registered new tserver with Master: 84e939a6df4f4f148d10263f4f7e0a15 (127.5.45.1:33435)
I20260807 15:54:17.972045 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 15:54:17.973680 6067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36978
I20260807 15:54:17.976091 6246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:17.976958 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:54:17.977304 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "0976de1ffe0f4c66b303bbf70baf3098"
format_stamp: "Formatted at 2026-08-07 15:54:17 on dist-test-slave-bw88"
I20260807 15:54:17.977686 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:18.019191 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:18.020717 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:18.022589 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:18.025128 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:18.025390 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 15:54:18.025662 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:18.025825 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:18.065049 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:45427
I20260807 15:54:18.065146 6322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:45427 every 8 connection(s)
I20260807 15:54:18.077575 6231 heartbeater.cc:503] Master 127.5.45.62:39023 was elected leader, sending a full tablet report...
I20260807 15:54:18.080572 6323 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39023
I20260807 15:54:18.080966 6323 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:18.081733 6323 heartbeater.cc:511] Master 127.5.45.62:39023 requested a full tablet report, sending...
I20260807 15:54:18.083388 6067 ts_manager.cc:194] Registered new tserver with Master: 0976de1ffe0f4c66b303bbf70baf3098 (127.5.45.2:45427)
I20260807 15:54:18.084044 5300 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015741906s
I20260807 15:54:18.085048 6067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36980
I20260807 15:54:18.107115 6067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:54:18.108582 6067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260807 15:54:18.145180 6277 tablet_service.cc:1467] Processing CreateTablet for tablet e9d8f03bd3144ae98541bfe24caad20e (DEFAULT_TABLE table=test-table [id=c1445fdedc97475eb94a7cdf89cfcf5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:18.146458 6277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9d8f03bd3144ae98541bfe24caad20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:18.147050 6183 tablet_service.cc:1467] Processing CreateTablet for tablet e9d8f03bd3144ae98541bfe24caad20e (DEFAULT_TABLE table=test-table [id=c1445fdedc97475eb94a7cdf89cfcf5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:18.147990 6183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9d8f03bd3144ae98541bfe24caad20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:18.157802 6340 tablet_bootstrap.cc:492] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: Bootstrap starting.
I20260807 15:54:18.159327 6341 tablet_bootstrap.cc:492] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: Bootstrap starting.
I20260807 15:54:18.163905 6340 tablet_bootstrap.cc:654] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:18.165242 6341 tablet_bootstrap.cc:654] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:18.169641 6340 tablet_bootstrap.cc:492] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: No bootstrap required, opened a new log
I20260807 15:54:18.170053 6340 ts_tablet_manager.cc:1397] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.001s
I20260807 15:54:18.171137 6341 tablet_bootstrap.cc:492] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: No bootstrap required, opened a new log
I20260807 15:54:18.171561 6341 ts_tablet_manager.cc:1397] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 15:54:18.172449 6340 raft_consensus.cc:348] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.173130 6340 raft_consensus.cc:729] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0976de1ffe0f4c66b303bbf70baf3098, State: Initialized, Role: FOLLOWER
I20260807 15:54:18.173880 6341 raft_consensus.cc:348] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.173727 6340 consensus_queue.cc:260] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.174490 6341 raft_consensus.cc:729] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Initialized, Role: FOLLOWER
I20260807 15:54:18.175088 6341 consensus_queue.cc:260] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.176067 6323 heartbeater.cc:503] Master 127.5.45.62:39023 was elected leader, sending a full tablet report...
I20260807 15:54:18.176642 6340 ts_tablet_manager.cc:1428] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260807 15:54:18.177912 6341 ts_tablet_manager.cc:1428] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 15:54:18.210178 5300 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 15:54:18.210464 5300 raft_consensus.cc:412] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 0 FOLLOWER]: Emulating election...
I20260807 15:54:18.210698 5300 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:18.214649 5300 raft_consensus.cc:686] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 1 LEADER]: Becoming Leader. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Running, Role: LEADER
W20260807 15:54:18.215026 6231 heartbeater.cc:644] Failed to heartbeat to 127.5.45.62:39023 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260807 15:54:18.215315 5300 consensus_queue.cc:237] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.218281 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:18.239279 5300 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 15:54:18.343037 5300 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 15:54:18.547228 5300 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260807 15:54:18.698370 6291 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:18.703959 6345 consensus_queue.cc:1035] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:18.852530 5300 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 15:54:18.852766 5300 raft_consensus.cc:412] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 1 LEADER]: Emulating election...
I20260807 15:54:18.852936 5300 raft_consensus.cc:3032] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:18.853083 5300 raft_consensus.cc:729] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Running, Role: LEADER
I20260807 15:54:18.853499 5300 consensus_queue.cc:260] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.854101 5300 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:18.857933 5300 raft_consensus.cc:686] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 2 LEADER]: Becoming Leader. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Running, Role: LEADER
I20260807 15:54:18.858474 5300 consensus_queue.cc:237] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:18.859925 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:18.863520 5300 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 15:54:18.968189 5300 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 15:54:19.172297 5300 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260807 15:54:19.288659 6291 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:19.293615 6291 raft_consensus.cc:1261] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 2 FOLLOWER]: Refusing update from remote peer 84e939a6df4f4f148d10263f4f7e0a15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:54:19.294986 6354 consensus_queue.cc:1035] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:19.479826 5300 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260807 15:54:19.480243 5300 raft_consensus.cc:412] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 2 LEADER]: Emulating election...
I20260807 15:54:19.480584 5300 raft_consensus.cc:3032] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 15:54:19.480863 5300 raft_consensus.cc:729] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Running, Role: LEADER
I20260807 15:54:19.481444 5300 consensus_queue.cc:260] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:19.482548 5300 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:19.488706 5300 raft_consensus.cc:686] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 3 LEADER]: Becoming Leader. State: Replica: 84e939a6df4f4f148d10263f4f7e0a15, State: Running, Role: LEADER
I20260807 15:54:19.489411 5300 consensus_queue.cc:237] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 } } peers { permanent_uuid: "84e939a6df4f4f148d10263f4f7e0a15" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33435 } }
I20260807 15:54:19.491024 5300 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260807 15:54:19.494930 5300 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 15:54:19.599011 5300 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 15:54:19.805043 5300 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260807 15:54:19.882575 6291 raft_consensus.cc:3037] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:19.887745 6291 raft_consensus.cc:1261] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 3 FOLLOWER]: Refusing update from remote peer 84e939a6df4f4f148d10263f4f7e0a15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 15:54:19.889485 6358 consensus_queue.cc:1035] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0976de1ffe0f4c66b303bbf70baf3098" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 15:54:20.113818 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:54:20.138945 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:20.139652 5300 tablet_replica.cc:330] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15: stopping tablet replica
I20260807 15:54:20.140376 5300 raft_consensus.cc:2229] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 3 LEADER]: Raft consensus shutting down.
I20260807 15:54:20.140975 5300 raft_consensus.cc:2258] T e9d8f03bd3144ae98541bfe24caad20e P 84e939a6df4f4f148d10263f4f7e0a15 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:20.161039 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:54:20.170967 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:54:20.196166 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:20.196755 5300 tablet_replica.cc:330] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098: stopping tablet replica
I20260807 15:54:20.197263 5300 raft_consensus.cc:2229] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 15:54:20.197777 5300 raft_consensus.cc:2258] T e9d8f03bd3144ae98541bfe24caad20e P 0976de1ffe0f4c66b303bbf70baf3098 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:20.217851 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:54:20.226419 5300 master.cc:537] Master@127.5.45.62:39023 shutting down...
I20260807 15:54:20.247355 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:20.247997 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:20.248348 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P f27e21e80bef41a299393e375075e630: stopping tablet replica
I20260807 15:54:20.265969 5300 master.cc:552] Master@127.5.45.62:39023 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2686 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10940 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260807 15:54:20.297482 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:40179
I20260807 15:54:20.298614 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:20.304216 6372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:20.305244 6373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:20.306845 6375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.307773 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:20.308560 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:20.308759 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:20.308961 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118060308928 us; error 0 us; skew 500 ppm
I20260807 15:54:20.309659 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:20.312427 5300 webserver.cc:460] Webserver started at http://127.5.45.62:46745/ using document root <none> and password file <none>
I20260807 15:54:20.312990 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:20.313205 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:20.313719 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:20.315042 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "56b006aa0659412f9026cce76f6602e4"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.319965 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:54:20.323625 6384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.324502 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:54:20.324755 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "56b006aa0659412f9026cce76f6602e4"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.325002 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:20.350098 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:20.351336 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:20.392305 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:40179
I20260807 15:54:20.392398 6444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:40179 every 8 connection(s)
I20260807 15:54:20.396600 6445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:20.406208 6445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4: Bootstrap starting.
I20260807 15:54:20.410609 6445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:20.415094 6445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4: No bootstrap required, opened a new log
I20260807 15:54:20.417091 6445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER }
I20260807 15:54:20.417567 6445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:20.417809 6445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b006aa0659412f9026cce76f6602e4, State: Initialized, Role: FOLLOWER
I20260807 15:54:20.418427 6445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER }
I20260807 15:54:20.418885 6445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:54:20.419107 6445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:54:20.419376 6445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:20.423784 6445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER }
I20260807 15:54:20.424314 6445 leader_election.cc:302] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56b006aa0659412f9026cce76f6602e4; no voters:
I20260807 15:54:20.425778 6445 leader_election.cc:288] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:54:20.426192 6449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:20.427680 6449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 1 LEADER]: Becoming Leader. State: Replica: 56b006aa0659412f9026cce76f6602e4, State: Running, Role: LEADER
I20260807 15:54:20.428409 6449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER }
I20260807 15:54:20.429273 6445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:20.431633 6451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56b006aa0659412f9026cce76f6602e4. Latest consensus state: current_term: 1 leader_uuid: "56b006aa0659412f9026cce76f6602e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER } }
I20260807 15:54:20.431685 6450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56b006aa0659412f9026cce76f6602e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b006aa0659412f9026cce76f6602e4" member_type: VOTER } }
I20260807 15:54:20.432302 6451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:20.432536 6450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:20.437628 6456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:20.445560 6456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:20.446588 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:20.454464 6456 catalog_manager.cc:1347] Generated new cluster ID: d407a9898ef140868d53cf4defe3eddd
I20260807 15:54:20.454843 6456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:20.471388 6456 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:20.473037 6456 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:20.486475 6456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4: Generated new TSK 0
I20260807 15:54:20.487221 6456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:20.513002 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:20.519416 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:20.520455 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.522966 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:20.522895 6475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.524024 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:20.524241 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:20.524410 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118060524396 us; error 0 us; skew 500 ppm
I20260807 15:54:20.524917 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:20.527689 5300 webserver.cc:460] Webserver started at http://127.5.45.1:36325/ using document root <none> and password file <none>
I20260807 15:54:20.528187 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:20.528391 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:20.528666 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:20.530089 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.534924 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:54:20.538622 6480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.539470 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:54:20.539934 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.540262 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:20.559614 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:20.560884 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:20.562521 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:20.564953 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:20.565188 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.565472 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:20.565735 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.607434 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46453
I20260807 15:54:20.607537 6563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46453 every 8 connection(s)
I20260807 15:54:20.612403 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:20.620177 6570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:20.622347 6571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.625850 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:20.626807 6574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.628167 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:20.628482 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:20.628710 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118060628670 us; error 0 us; skew 500 ppm
I20260807 15:54:20.629490 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:20.629606 6564 heartbeater.cc:347] Connected to a master server at 127.5.45.62:40179
I20260807 15:54:20.630051 6564 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:20.630934 6564 heartbeater.cc:511] Master 127.5.45.62:40179 requested a full tablet report, sending...
I20260807 15:54:20.632781 5300 webserver.cc:460] Webserver started at http://127.5.45.2:36027/ using document root <none> and password file <none>
I20260807 15:54:20.633474 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:20.633390 6406 ts_manager.cc:194] Registered new tserver with Master: 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa (127.5.45.1:46453)
I20260807 15:54:20.633780 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:20.634130 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:20.635799 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "b48c9068d4914f99a2b4d9a67a212f30"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.636241 6406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56048
I20260807 15:54:20.641296 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 15:54:20.644902 6580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.645851 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:54:20.646188 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "b48c9068d4914f99a2b4d9a67a212f30"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.646487 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:20.664307 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:20.665711 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:20.667223 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:20.669528 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:20.669714 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.669988 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:20.670146 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.710726 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:45517
I20260807 15:54:20.710834 6654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:45517 every 8 connection(s)
I20260807 15:54:20.715457 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:20.723428 6658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:20.724058 6659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.727113 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:20.727805 6662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:20.728809 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:20.729054 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:20.729234 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118060729222 us; error 0 us; skew 500 ppm
I20260807 15:54:20.729826 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:20.731837 6655 heartbeater.cc:347] Connected to a master server at 127.5.45.62:40179
I20260807 15:54:20.732133 6655 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:20.732810 6655 heartbeater.cc:511] Master 127.5.45.62:40179 requested a full tablet report, sending...
I20260807 15:54:20.732959 5300 webserver.cc:460] Webserver started at http://127.5.45.3:41701/ using document root <none> and password file <none>
I20260807 15:54:20.733649 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:20.733892 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:20.734215 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:20.735116 6406 ts_manager.cc:194] Registered new tserver with Master: b48c9068d4914f99a2b4d9a67a212f30 (127.5.45.2:45517)
I20260807 15:54:20.735728 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-2-root/instance:
uuid: "431b63950af545d69919f78cd5511f74"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.737301 6406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56050
I20260807 15:54:20.741273 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 15:54:20.744688 6668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.745622 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:54:20.745910 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-2-root
uuid: "431b63950af545d69919f78cd5511f74"
format_stamp: "Formatted at 2026-08-07 15:54:20 on dist-test-slave-bw88"
I20260807 15:54:20.746198 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786118049188364-5300-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:20.759521 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:20.760767 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:20.762300 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:20.764519 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:20.764750 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.764999 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:20.765172 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:20.836721 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.3:33787
I20260807 15:54:20.836833 6746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.3:33787 every 8 connection(s)
I20260807 15:54:20.854600 6747 heartbeater.cc:347] Connected to a master server at 127.5.45.62:40179
I20260807 15:54:20.855007 6747 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:20.855858 6747 heartbeater.cc:511] Master 127.5.45.62:40179 requested a full tablet report, sending...
I20260807 15:54:20.857867 6406 ts_manager.cc:194] Registered new tserver with Master: 431b63950af545d69919f78cd5511f74 (127.5.45.3:33787)
I20260807 15:54:20.858850 5300 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017160562s
I20260807 15:54:20.859382 6406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56054
I20260807 15:54:20.886307 6406 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260807 15:54:20.887972 6406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:54:20.944087 6705 tablet_service.cc:1467] Processing CreateTablet for tablet bae90f3641384b0aabd11486d156a259 (DEFAULT_TABLE table=test-table [id=da37552e043b4b9c8c88136bf16d7cbb]), partition=
I20260807 15:54:20.945196 6705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae90f3641384b0aabd11486d156a259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:20.952404 6519 tablet_service.cc:1467] Processing CreateTablet for tablet bae90f3641384b0aabd11486d156a259 (DEFAULT_TABLE table=test-table [id=da37552e043b4b9c8c88136bf16d7cbb]), partition=
I20260807 15:54:20.953291 6519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae90f3641384b0aabd11486d156a259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:20.954886 6615 tablet_service.cc:1467] Processing CreateTablet for tablet bae90f3641384b0aabd11486d156a259 (DEFAULT_TABLE table=test-table [id=da37552e043b4b9c8c88136bf16d7cbb]), partition=
I20260807 15:54:20.955965 6615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae90f3641384b0aabd11486d156a259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:20.961475 6766 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: Bootstrap starting.
I20260807 15:54:20.967564 6769 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: Bootstrap starting.
I20260807 15:54:20.968276 6766 tablet_bootstrap.cc:654] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:20.972165 6771 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: Bootstrap starting.
I20260807 15:54:20.974965 6769 tablet_bootstrap.cc:654] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:20.976696 6766 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: No bootstrap required, opened a new log
I20260807 15:54:20.977182 6766 ts_tablet_manager.cc:1397] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260807 15:54:20.979310 6771 tablet_bootstrap.cc:654] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:20.980679 6766 raft_consensus.cc:348] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.981546 6766 raft_consensus.cc:374] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:20.981657 6769 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: No bootstrap required, opened a new log
I20260807 15:54:20.981870 6766 raft_consensus.cc:729] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431b63950af545d69919f78cd5511f74, State: Initialized, Role: FOLLOWER
I20260807 15:54:20.982319 6769 ts_tablet_manager.cc:1397] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260807 15:54:20.982700 6766 consensus_queue.cc:260] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.985080 6769 raft_consensus.cc:348] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.985800 6769 raft_consensus.cc:374] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:20.986130 6769 raft_consensus.cc:729] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa, State: Initialized, Role: FOLLOWER
I20260807 15:54:20.986896 6771 tablet_bootstrap.cc:492] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: No bootstrap required, opened a new log
I20260807 15:54:20.987206 6747 heartbeater.cc:503] Master 127.5.45.62:40179 was elected leader, sending a full tablet report...
I20260807 15:54:20.987408 6771 ts_tablet_manager.cc:1397] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260807 15:54:20.987521 6766 ts_tablet_manager.cc:1428] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260807 15:54:20.987011 6769 consensus_queue.cc:260] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.990466 6564 heartbeater.cc:503] Master 127.5.45.62:40179 was elected leader, sending a full tablet report...
I20260807 15:54:20.990970 6769 ts_tablet_manager.cc:1428] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:54:20.990490 6771 raft_consensus.cc:348] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.991816 6771 raft_consensus.cc:374] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:20.992102 6771 raft_consensus.cc:729] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48c9068d4914f99a2b4d9a67a212f30, State: Initialized, Role: FOLLOWER
I20260807 15:54:20.992813 6771 consensus_queue.cc:260] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:20.998924 6655 heartbeater.cc:503] Master 127.5.45.62:40179 was elected leader, sending a full tablet report...
I20260807 15:54:20.999259 6771 ts_tablet_manager.cc:1428] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 15:54:21.037102 6776 raft_consensus.cc:482] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:21.037758 6776 raft_consensus.cc:504] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:21.040031 6776 leader_election.cc:288] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b48c9068d4914f99a2b4d9a67a212f30 (127.5.45.2:45517), 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa (127.5.45.1:46453)
I20260807 15:54:21.047808 6777 raft_consensus.cc:482] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:21.048487 6777 raft_consensus.cc:504] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:21.057572 6777 leader_election.cc:288] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431b63950af545d69919f78cd5511f74 (127.5.45.3:33787), b48c9068d4914f99a2b4d9a67a212f30 (127.5.45.2:45517)
I20260807 15:54:21.058482 6627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "431b63950af545d69919f78cd5511f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b48c9068d4914f99a2b4d9a67a212f30" is_pre_election: true
I20260807 15:54:21.059567 6627 raft_consensus.cc:2454] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431b63950af545d69919f78cd5511f74 in term 0.
I20260807 15:54:21.060554 6531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "431b63950af545d69919f78cd5511f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" is_pre_election: true
I20260807 15:54:21.061286 6531 raft_consensus.cc:2454] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431b63950af545d69919f78cd5511f74 in term 0.
I20260807 15:54:21.062031 6670 leader_election.cc:302] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431b63950af545d69919f78cd5511f74, b48c9068d4914f99a2b4d9a67a212f30; no voters:
I20260807 15:54:21.063159 6776 raft_consensus.cc:2781] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:21.063508 6776 raft_consensus.cc:482] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:21.063890 6776 raft_consensus.cc:3037] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:21.069478 6718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431b63950af545d69919f78cd5511f74" is_pre_election: true
I20260807 15:54:21.069834 6776 raft_consensus.cc:504] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:21.071069 6718 raft_consensus.cc:2379] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa in current term 1: Already voted for candidate 431b63950af545d69919f78cd5511f74 in this term.
I20260807 15:54:21.072729 6627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "431b63950af545d69919f78cd5511f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b48c9068d4914f99a2b4d9a67a212f30"
I20260807 15:54:21.072458 6776 leader_election.cc:288] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [CANDIDATE]: Term 1 election: Requested vote from peers b48c9068d4914f99a2b4d9a67a212f30 (127.5.45.2:45517), 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa (127.5.45.1:46453)
I20260807 15:54:21.073114 6626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b48c9068d4914f99a2b4d9a67a212f30" is_pre_election: true
I20260807 15:54:21.073357 6627 raft_consensus.cc:3037] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:21.073556 6531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae90f3641384b0aabd11486d156a259" candidate_uuid: "431b63950af545d69919f78cd5511f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa"
I20260807 15:54:21.074136 6531 raft_consensus.cc:3037] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:21.074806 6482 leader_election.cc:302] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa; no voters: 431b63950af545d69919f78cd5511f74, b48c9068d4914f99a2b4d9a67a212f30
I20260807 15:54:21.080922 6627 raft_consensus.cc:2454] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431b63950af545d69919f78cd5511f74 in term 1.
I20260807 15:54:21.080922 6531 raft_consensus.cc:2454] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431b63950af545d69919f78cd5511f74 in term 1.
I20260807 15:54:21.081869 6777 raft_consensus.cc:2726] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 15:54:21.082381 6670 leader_election.cc:302] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431b63950af545d69919f78cd5511f74, b48c9068d4914f99a2b4d9a67a212f30; no voters:
I20260807 15:54:21.083487 6776 raft_consensus.cc:2781] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:21.086244 6776 raft_consensus.cc:686] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 LEADER]: Becoming Leader. State: Replica: 431b63950af545d69919f78cd5511f74, State: Running, Role: LEADER
I20260807 15:54:21.087131 6776 consensus_queue.cc:237] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } }
I20260807 15:54:21.096536 6404 catalog_manager.cc:5495] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 431b63950af545d69919f78cd5511f74 (127.5.45.3). New cstate: current_term: 1 leader_uuid: "431b63950af545d69919f78cd5511f74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431b63950af545d69919f78cd5511f74" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:21.535372 6776 consensus_queue.cc:1035] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b48c9068d4914f99a2b4d9a67a212f30" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:21.549052 6784 consensus_queue.cc:1035] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d3d5c6d5c6b4fe6a71092c3e7e1c2aa" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 15:54:22.723925 5300 tablet_replica.cc:330] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa: stopping tablet replica
I20260807 15:54:22.724521 5300 raft_consensus.cc:2229] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:54:22.724989 5300 raft_consensus.cc:2258] T bae90f3641384b0aabd11486d156a259 P 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:54:23.128834 6651 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc400 after lost signal to thread 5304
W20260807 15:54:23.129797 6651 debug-util.cc:398] Leaking SignalData structure 0x7b08000dcb60 after lost signal to thread 6444
W20260807 15:54:23.130689 6651 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3420 after lost signal to thread 6563
W20260807 15:54:23.130925 6651 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2c80 after lost signal to thread 6569
W20260807 15:54:23.131520 6651 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3120 after lost signal to thread 6654
W20260807 15:54:23.132165 6651 debug-util.cc:398] Leaking SignalData structure 0x7b0800061040 after lost signal to thread 6746
I20260807 15:54:23.151822 6669 consensus_queue.cc:579] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [LEADER]: The tablet replica hosted on peer 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa has failed
W20260807 15:54:23.152451 6669 consensus_peers.cc:487] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 -> Peer 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa (127.5.45.1:46453): Couldn't send request to peer 1d3d5c6d5c6b4fe6a71092c3e7e1c2aa. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:54:23.251382 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:54:23.277519 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:23.296062 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:54:23.305193 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:54:23.330817 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:23.331426 5300 tablet_replica.cc:330] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30: stopping tablet replica
I20260807 15:54:23.331959 5300 raft_consensus.cc:2229] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:54:23.332387 5300 raft_consensus.cc:2258] T bae90f3641384b0aabd11486d156a259 P b48c9068d4914f99a2b4d9a67a212f30 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:54:23.444242 6670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.45.2:45517: connect: Connection refused (error 111)
W20260807 15:54:23.447762 6670 consensus_peers.cc:487] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 -> Peer b48c9068d4914f99a2b4d9a67a212f30 (127.5.45.2:45517): Couldn't send request to peer b48c9068d4914f99a2b4d9a67a212f30. Status: Network error: Client connection negotiation failed: client connection to 127.5.45.2:45517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 15:54:24.349275 5300 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 6651)
I20260807 15:54:25.265317 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:54:25.273699 5300 tablet_server.cc:178] TabletServer@127.5.45.3:0 shutting down...
I20260807 15:54:25.293035 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:25.293653 5300 tablet_replica.cc:330] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74: stopping tablet replica
I20260807 15:54:25.294162 5300 raft_consensus.cc:2229] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:25.294909 5300 raft_consensus.cc:2258] T bae90f3641384b0aabd11486d156a259 P 431b63950af545d69919f78cd5511f74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:25.313128 5300 tablet_server.cc:194] TabletServer@127.5.45.3:0 shutdown complete.
I20260807 15:54:25.321640 5300 master.cc:537] Master@127.5.45.62:40179 shutting down...
I20260807 15:54:25.339453 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:25.339994 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:25.340322 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56b006aa0659412f9026cce76f6602e4: stopping tablet replica
I20260807 15:54:25.357769 5300 master.cc:552] Master@127.5.45.62:40179 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (5085 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260807 15:54:25.382953 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:46693
I20260807 15:54:25.384007 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:25.388881 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.390205 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.391701 6809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:25.392560 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:25.393383 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:25.393584 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:25.393736 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118065393723 us; error 0 us; skew 500 ppm
I20260807 15:54:25.394520 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:25.400554 5300 webserver.cc:460] Webserver started at http://127.5.45.62:38143/ using document root <none> and password file <none>
I20260807 15:54:25.401072 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:25.401283 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:25.401908 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:25.403035 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.407725 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:54:25.411233 6814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.412123 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:54:25.412361 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.412601 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:25.432169 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:25.433173 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:25.469494 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:46693
I20260807 15:54:25.469548 6877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:46693 every 8 connection(s)
I20260807 15:54:25.473229 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:25.482723 6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396: Bootstrap starting.
I20260807 15:54:25.486730 6878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:25.490530 6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396: No bootstrap required, opened a new log
I20260807 15:54:25.492259 6878 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER }
I20260807 15:54:25.492630 6878 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:25.492800 6878 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb0f12d2c9d4c69a746b37fcaa3e396, State: Initialized, Role: FOLLOWER
I20260807 15:54:25.493399 6878 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER }
I20260807 15:54:25.493842 6878 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:54:25.494043 6878 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:54:25.494239 6878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:25.498243 6878 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER }
I20260807 15:54:25.498786 6878 leader_election.cc:302] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddb0f12d2c9d4c69a746b37fcaa3e396; no voters:
I20260807 15:54:25.499985 6878 leader_election.cc:288] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:54:25.500277 6882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:25.501607 6882 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 1 LEADER]: Becoming Leader. State: Replica: ddb0f12d2c9d4c69a746b37fcaa3e396, State: Running, Role: LEADER
I20260807 15:54:25.502255 6882 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER }
I20260807 15:54:25.503019 6878 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:25.506624 6883 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER } }
I20260807 15:54:25.507650 6883 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:25.507376 6884 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddb0f12d2c9d4c69a746b37fcaa3e396. Latest consensus state: current_term: 1 leader_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb0f12d2c9d4c69a746b37fcaa3e396" member_type: VOTER } }
I20260807 15:54:25.509778 6884 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:25.511651 6892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:25.517328 6892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:25.519631 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:25.525853 6892 catalog_manager.cc:1347] Generated new cluster ID: a5c831f3f3964d65bb413739e7f075ce
I20260807 15:54:25.526105 6892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:25.540405 6892 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:25.542167 6892 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:25.561816 6892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396: Generated new TSK 0
I20260807 15:54:25.562453 6892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:25.586102 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:25.591948 6904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.593291 6906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.595184 6908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:25.595563 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:25.596362 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:25.596536 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:25.596724 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118065596711 us; error 0 us; skew 500 ppm
I20260807 15:54:25.597285 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:25.599711 5300 webserver.cc:460] Webserver started at http://127.5.45.1:46055/ using document root <none> and password file <none>
I20260807 15:54:25.600198 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:25.600384 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:25.600685 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:25.601869 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "ff66168765fb4b978adadefcaf4b2582"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.606590 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:54:25.610006 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.610795 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:54:25.611086 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "ff66168765fb4b978adadefcaf4b2582"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.611390 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:25.635267 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:25.636475 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:25.638027 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:25.640293 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:25.640516 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.640759 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:25.640918 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.680011 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46291
I20260807 15:54:25.680091 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46291 every 8 connection(s)
I20260807 15:54:25.695983 6989 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46693
I20260807 15:54:25.696368 6989 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:25.697031 6989 heartbeater.cc:511] Master 127.5.45.62:46693 requested a full tablet report, sending...
I20260807 15:54:25.698881 6839 ts_manager.cc:194] Registered new tserver with Master: ff66168765fb4b978adadefcaf4b2582 (127.5.45.1:46291)
I20260807 15:54:25.699898 5300 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016560282s
I20260807 15:54:25.700747 6839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38430
I20260807 15:54:25.721782 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:54:25.736039 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:25.751777 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:54:25.759646 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:54:25.762224 5300 tablet_server.cc:93] deduplicated master addresses: 127.5.45.62:46693
W20260807 15:54:25.765743 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.766438 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.766749 7005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:25.768707 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:25.769567 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:25.769747 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:25.769858 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118065769850 us; error 0 us; skew 500 ppm
I20260807 15:54:25.770295 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:25.772583 5300 webserver.cc:460] Webserver started at http://127.5.45.1:46055/ using document root <none> and password file <none>
I20260807 15:54:25.773025 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:25.773160 5300 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 15:54:25.776585 5300 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:54:25.779209 7010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.779961 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:54:25.780241 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "ff66168765fb4b978adadefcaf4b2582"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.780536 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:25.799868 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:25.800909 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:25.802425 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:25.804899 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:25.805122 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.805384 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:25.805555 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 15:54:25.843997 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46291
I20260807 15:54:25.844123 7090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46291 every 8 connection(s)
I20260807 15:54:25.849936 5300 tablet_server.cc:178] TabletServer@127.5.45.1:46291 shutting down...
I20260807 15:54:25.870424 7092 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46693
I20260807 15:54:25.871013 7092 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:25.872031 7092 heartbeater.cc:511] Master 127.5.45.62:46693 requested a full tablet report, sending...
I20260807 15:54:25.874174 6839 ts_manager.cc:194] Re-registered known tserver with Master: ff66168765fb4b978adadefcaf4b2582 (127.5.45.1:46291)
I20260807 15:54:25.875655 6839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38448
I20260807 15:54:25.878284 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:54:25.894364 5300 tablet_server.cc:194] TabletServer@127.5.45.1:46291 shutdown complete.
I20260807 15:54:25.901854 5300 master.cc:537] Master@127.5.45.62:46693 shutting down...
I20260807 15:54:25.917315 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:54:25.917840 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:25.918191 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P ddb0f12d2c9d4c69a746b37fcaa3e396: stopping tablet replica
I20260807 15:54:25.936614 5300 master.cc:552] Master@127.5.45.62:46693 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (575 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260807 15:54:25.958113 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:46661,127.5.45.61:36549,127.5.45.60:42447
I20260807 15:54:25.959354 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:25.964025 7100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:25.965286 7101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:25.967026 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:25.967700 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:25.968442 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:25.968650 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:25.968808 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118065968793 us; error 0 us; skew 500 ppm
I20260807 15:54:25.969316 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:25.971732 5300 webserver.cc:460] Webserver started at http://127.5.45.62:40897/ using document root <none> and password file <none>
I20260807 15:54:25.972226 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:25.972426 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:25.972692 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:25.973896 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "4dc450c6e3db439994750939886f3aa7"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.978317 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 15:54:25.981688 7110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:25.982419 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:54:25.982712 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "4dc450c6e3db439994750939886f3aa7"
format_stamp: "Formatted at 2026-08-07 15:54:25 on dist-test-slave-bw88"
I20260807 15:54:25.982999 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:25.993588 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:25.994686 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.032223 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:46661
I20260807 15:54:26.032352 7174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:46661 every 8 connection(s)
I20260807 15:54:26.036037 7176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:26.036387 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:26.041886 7178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.042810 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } has no permanent_uuid. Determining permanent_uuid...
W20260807 15:54:26.042881 7179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:26.044836 7181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.046444 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:26.047394 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:26.047623 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:26.047809 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118066047794 us; error 0 us; skew 500 ppm
I20260807 15:54:26.048514 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:26.051182 5300 webserver.cc:460] Webserver started at http://127.5.45.61:37799/ using document root <none> and password file <none>
I20260807 15:54:26.051795 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:26.052050 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:26.052353 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:26.053682 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-1-root/instance:
uuid: "df6a37abab2d49529a2a2ca42d10c7cf"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.059427 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 15:54:26.059427 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:54:26.063867 7193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.064765 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:54:26.065062 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-1-root
uuid: "df6a37abab2d49529a2a2ca42d10c7cf"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
W20260807 15:54:26.065148 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.61:36549: Network error: Client connection negotiation failed: client connection to 127.5.45.61:36549: connect: Connection refused (error 111)
I20260807 15:54:26.065482 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:26.087920 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:26.088944 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.110809 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } attempt: 1
W20260807 15:54:26.115983 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.61:36549: Network error: Client connection negotiation failed: client connection to 127.5.45.61:36549: connect: Connection refused (error 111)
I20260807 15:54:26.127596 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.61:36549
I20260807 15:54:26.127727 7267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.61:36549 every 8 connection(s)
I20260807 15:54:26.131345 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:54:26.131707 7268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 15:54:26.137075 7270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:26.137529 7271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.139962 7268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } has no permanent_uuid. Determining permanent_uuid...
W20260807 15:54:26.140590 7273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.140959 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:26.142139 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:26.142474 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:26.142706 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118066142693 us; error 0 us; skew 500 ppm
I20260807 15:54:26.143253 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:26.146373 5300 webserver.cc:460] Webserver started at http://127.5.45.60:43949/ using document root <none> and password file <none>
I20260807 15:54:26.146925 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:26.147169 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:26.147476 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:26.148872 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root/instance:
uuid: "a3acfcc965f048a4908d9687c07fd330"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.150484 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } attempt: 2
I20260807 15:54:26.155268 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 15:54:26.156289 7268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:54:26.160681 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.162173 5300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:54:26.162623 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
uuid: "a3acfcc965f048a4908d9687c07fd330"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.163069 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:26.166460 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
W20260807 15:54:26.171036 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.60:42447: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:26.179657 7268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
W20260807 15:54:26.184434 7268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.60:42447: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:26.187287 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:26.188223 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.208997 7268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } attempt: 1
I20260807 15:54:26.209470 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } attempt: 1
W20260807 15:54:26.214995 7268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.60:42447: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
W20260807 15:54:26.214995 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.45.60:42447: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:26.227082 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.60:42447
I20260807 15:54:26.227190 7349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.60:42447 every 8 connection(s)
I20260807 15:54:26.230150 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:54:26.230825 7350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:26.236322 7350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:54:26.248207 7350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:54:26.258600 7350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:54:26.264668 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } attempt: 2
I20260807 15:54:26.266599 7268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } attempt: 2
I20260807 15:54:26.278352 7350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Bootstrap starting.
I20260807 15:54:26.284260 7350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:26.287447 7176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: Bootstrap starting.
I20260807 15:54:26.288705 7350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: No bootstrap required, opened a new log
I20260807 15:54:26.290916 7268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: Bootstrap starting.
I20260807 15:54:26.290727 7350 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.291311 7350 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:26.291610 7350 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3acfcc965f048a4908d9687c07fd330, State: Initialized, Role: FOLLOWER
I20260807 15:54:26.292260 7350 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.293314 7176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:26.294322 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.295404 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.296007 7350 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:26.297079 7268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:26.298919 7176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: No bootstrap required, opened a new log
I20260807 15:54:26.302347 7176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.303369 7176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:26.303745 7176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc450c6e3db439994750939886f3aa7, State: Initialized, Role: FOLLOWER
I20260807 15:54:26.304615 7176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.307384 7371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.308507 7371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.308768 7268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: No bootstrap required, opened a new log
I20260807 15:54:26.308919 7176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:26.311637 7268 raft_consensus.cc:348] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.312428 7268 raft_consensus.cc:374] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:26.312976 7268 raft_consensus.cc:729] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6a37abab2d49529a2a2ca42d10c7cf, State: Initialized, Role: FOLLOWER
I20260807 15:54:26.313968 7268 consensus_queue.cc:260] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.316821 7385 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:26.317435 7385 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.321424 7147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7" is_pre_election: true
I20260807 15:54:26.321667 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:26.322376 7147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 0.
I20260807 15:54:26.322615 7317 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 0.
I20260807 15:54:26.324070 7199 leader_election.cc:302] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf; no voters:
I20260807 15:54:26.324373 7385 leader_election.cc:288] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:26.325218 7385 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.326066 7385 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.326195 7386 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:54:26.326555 7385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:26.326900 7385 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:26.327281 7385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:26.332350 7268 sys_catalog.cc:564] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:26.333024 7385 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.335835 7147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7"
I20260807 15:54:26.336067 7385 leader_election.cc:288] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 1 election: Requested vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:26.336447 7147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:26.337289 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330"
I20260807 15:54:26.337873 7317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:26.343448 7147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 1.
I20260807 15:54:26.344631 7199 leader_election.cc:302] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf; no voters:
I20260807 15:54:26.345566 7317 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 1.
I20260807 15:54:26.346325 7385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:26.347054 7385 raft_consensus.cc:686] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 LEADER]: Becoming Leader. State: Replica: df6a37abab2d49529a2a2ca42d10c7cf, State: Running, Role: LEADER
I20260807 15:54:26.347712 7385 consensus_queue.cc:237] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:26.351876 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.352697 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:26.355937 7405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:26.361071 7405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:26.363686 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260807 15:54:26.368206 7411 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:54:26.368587 7411 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:54:26.370020 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260807 15:54:26.371884 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:54:26.372344 7147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Refusing update from remote peer df6a37abab2d49529a2a2ca42d10c7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:54:26.372344 7317 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Refusing update from remote peer df6a37abab2d49529a2a2ca42d10c7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:54:26.374051 7385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:26.374610 7391 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:54:26.389575 7418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.389596 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.390705 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.394557 7413 mvcc.cc:205] Tried to move back new op lower bound from 7315939599846780928 to 7315939599767252992. Current Snapshot: MvccSnapshot[applied={T|T < 7315939599846780928}]
W20260807 15:54:26.395603 7422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.405187 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.406184 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.407090 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:26.408150 7431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.409205 7371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.409485 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 15:54:26.409885 7371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:54:26.409943 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:26.410252 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118066410233 us; error 0 us; skew 500 ppm
I20260807 15:54:26.411303 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:26.415463 7405 catalog_manager.cc:1347] Generated new cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:26.415795 7405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:26.416435 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.416687 7387 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.417459 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:26.417542 7387 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:26.421932 7371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:26.422719 7371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:26.424311 5300 webserver.cc:460] Webserver started at http://127.5.45.1:38349/ using document root <none> and password file <none>
I20260807 15:54:26.424875 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:26.425087 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:26.425432 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:26.426608 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "f07486fcec2343278bcc737786ff6fed"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.431980 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 15:54:26.436131 7439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.437073 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:54:26.437415 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "f07486fcec2343278bcc737786ff6fed"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.437721 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:26.456847 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:26.458261 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.459357 7405 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:54:26.461781 7405 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:26.464933 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:26.467903 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:26.468118 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.468382 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:26.468531 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.487130 7405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: Generated new TSK 0
I20260807 15:54:26.487975 7405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:26.523716 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:42027
I20260807 15:54:26.523818 7518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:42027 every 8 connection(s)
I20260807 15:54:26.538298 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:26.551482 7526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:26.554706 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.560156 5300 server_base.cc:1034] running on GCE node
W20260807 15:54:26.562971 7530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.564215 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:26.564486 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:26.564692 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118066564673 us; error 0 us; skew 500 ppm
I20260807 15:54:26.565393 7519 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:26.565778 7520 heartbeater.cc:347] Connected to a master server at 127.5.45.61:36549
I20260807 15:54:26.565838 7519 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.566180 7520 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.566817 7519 heartbeater.cc:511] Master 127.5.45.60:42447 requested a full tablet report, sending...
I20260807 15:54:26.566951 7520 heartbeater.cc:511] Master 127.5.45.61:36549 requested a full tablet report, sending...
I20260807 15:54:26.565428 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:26.569324 7226 ts_manager.cc:194] Registered new tserver with Master: f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027)
I20260807 15:54:26.569938 7305 ts_manager.cc:194] Registered new tserver with Master: f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027)
I20260807 15:54:26.572017 7226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56944
I20260807 15:54:26.573268 5300 webserver.cc:460] Webserver started at http://127.5.45.2:38259/ using document root <none> and password file <none>
I20260807 15:54:26.573518 7521 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46661
I20260807 15:54:26.573881 7521 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.573958 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:26.574272 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:26.574599 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:26.575155 7521 heartbeater.cc:511] Master 127.5.45.62:46661 requested a full tablet report, sending...
I20260807 15:54:26.576149 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "f847539a423f4000a7839152f3a1fa99"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.577946 7134 ts_manager.cc:194] Registered new tserver with Master: f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027)
I20260807 15:54:26.581200 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 15:54:26.584885 7536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.585664 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:54:26.585954 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "f847539a423f4000a7839152f3a1fa99"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.586252 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:26.605719 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:26.607014 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.609387 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:26.612427 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:26.612712 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.612979 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:26.613157 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.675697 7520 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:26.702562 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:37981
I20260807 15:54:26.702677 7618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:37981 every 8 connection(s)
I20260807 15:54:26.717825 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:54:26.728350 7626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:26.730881 7627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.736871 7619 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:26.737471 7619 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.738747 7619 heartbeater.cc:511] Master 127.5.45.60:42447 requested a full tablet report, sending...
I20260807 15:54:26.739104 7622 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46661
I20260807 15:54:26.739634 7622 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.740733 7622 heartbeater.cc:511] Master 127.5.45.62:46661 requested a full tablet report, sending...
I20260807 15:54:26.742509 7305 ts_manager.cc:194] Registered new tserver with Master: f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:26.743934 7134 ts_manager.cc:194] Registered new tserver with Master: f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
W20260807 15:54:26.745093 7629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:26.746464 7620 heartbeater.cc:347] Connected to a master server at 127.5.45.61:36549
I20260807 15:54:26.746832 7620 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.747509 7620 heartbeater.cc:511] Master 127.5.45.61:36549 requested a full tablet report, sending...
I20260807 15:54:26.749460 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:26.749866 7226 ts_manager.cc:194] Registered new tserver with Master: f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:26.750362 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:26.750593 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:26.750767 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118066750755 us; error 0 us; skew 500 ppm
I20260807 15:54:26.751323 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:26.751660 7226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56958
I20260807 15:54:26.754429 5300 webserver.cc:460] Webserver started at http://127.5.45.3:34337/ using document root <none> and password file <none>
I20260807 15:54:26.755002 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:26.755227 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:54:26.755499 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:54:26.756561 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-2-root/instance:
uuid: "d77851634f654ce5bfce79a4dd797055"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.761546 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 15:54:26.765100 7634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.765990 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:54:26.766338 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-2-root
uuid: "d77851634f654ce5bfce79a4dd797055"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:26.766669 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:54:26.792259 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:26.793593 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:26.794787 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:54:26.796996 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:54:26.797221 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.797520 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:54:26.797748 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:26.854632 7620 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:26.877295 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.3:34107
I20260807 15:54:26.877456 7712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.3:34107 every 8 connection(s)
I20260807 15:54:26.908273 7713 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:26.908740 7713 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.909705 7713 heartbeater.cc:511] Master 127.5.45.60:42447 requested a full tablet report, sending...
I20260807 15:54:26.909847 7714 heartbeater.cc:347] Connected to a master server at 127.5.45.61:36549
I20260807 15:54:26.910292 7714 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.911190 7714 heartbeater.cc:511] Master 127.5.45.61:36549 requested a full tablet report, sending...
I20260807 15:54:26.912540 7305 ts_manager.cc:194] Registered new tserver with Master: d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:26.913501 7226 ts_manager.cc:194] Registered new tserver with Master: d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:26.914515 7715 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46661
I20260807 15:54:26.914901 7715 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:26.915753 7715 heartbeater.cc:511] Master 127.5.45.62:46661 requested a full tablet report, sending...
I20260807 15:54:26.916397 7226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56974
I20260807 15:54:26.918514 7134 ts_manager.cc:194] Registered new tserver with Master: d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:26.919313 5300 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021525773s
I20260807 15:54:26.964118 7226 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260807 15:54:26.965845 7226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:54:27.019901 7714 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:27.048488 7575 tablet_service.cc:1467] Processing CreateTablet for tablet fcb581517aa04c7dbeba2ba716118eed (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.049268 7574 tablet_service.cc:1467] Processing CreateTablet for tablet 21d3feeebcad478aa30ccdde78044663 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.049614 7573 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2b29f18f024df199716835932639a1 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.050100 7575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb581517aa04c7dbeba2ba716118eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.051016 7573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2b29f18f024df199716835932639a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.052066 7574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d3feeebcad478aa30ccdde78044663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.055104 7674 tablet_service.cc:1467] Processing CreateTablet for tablet fcb581517aa04c7dbeba2ba716118eed (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.056131 7674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb581517aa04c7dbeba2ba716118eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.058420 7673 tablet_service.cc:1467] Processing CreateTablet for tablet 21d3feeebcad478aa30ccdde78044663 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.059737 7673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d3feeebcad478aa30ccdde78044663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.062207 7672 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2b29f18f024df199716835932639a1 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.063519 7672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2b29f18f024df199716835932639a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.063861 7471 tablet_service.cc:1467] Processing CreateTablet for tablet fcb581517aa04c7dbeba2ba716118eed (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.064204 7469 tablet_service.cc:1467] Processing CreateTablet for tablet 21d3feeebcad478aa30ccdde78044663 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.068733 7471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb581517aa04c7dbeba2ba716118eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.071673 7468 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2b29f18f024df199716835932639a1 (DEFAULT_TABLE table=test-table [id=549be6f3994e45de82118121367e9e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260807 15:54:27.072392 7469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d3feeebcad478aa30ccdde78044663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.075882 7468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2b29f18f024df199716835932639a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:54:27.128983 7738 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Bootstrap starting.
I20260807 15:54:27.133158 7737 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: Bootstrap starting.
I20260807 15:54:27.136994 7740 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: Bootstrap starting.
I20260807 15:54:27.141594 7738 tablet_bootstrap.cc:654] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.146559 7737 tablet_bootstrap.cc:654] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.149240 7740 tablet_bootstrap.cc:654] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.151615 7738 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: No bootstrap required, opened a new log
I20260807 15:54:27.152128 7738 ts_tablet_manager.cc:1397] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.004s
I20260807 15:54:27.154292 7737 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: No bootstrap required, opened a new log
I20260807 15:54:27.155272 7738 raft_consensus.cc:348] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.156088 7738 raft_consensus.cc:374] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.156421 7738 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.156554 7740 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: No bootstrap required, opened a new log
I20260807 15:54:27.157039 7740 ts_tablet_manager.cc:1397] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.000s
I20260807 15:54:27.154784 7737 ts_tablet_manager.cc:1397] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.007s
I20260807 15:54:27.158128 7738 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.160722 7740 raft_consensus.cc:348] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.161787 7740 raft_consensus.cc:374] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.162235 7740 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.161804 7737 raft_consensus.cc:348] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.163025 7737 raft_consensus.cc:374] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.163070 7738 ts_tablet_manager.cc:1428] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260807 15:54:27.163616 7737 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.164372 7738 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Bootstrap starting.
I20260807 15:54:27.164635 7737 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.170900 7738 tablet_bootstrap.cc:654] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.163077 7740 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.185688 7737 ts_tablet_manager.cc:1428] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: Time spent starting tablet: real 0.028s user 0.013s sys 0.000s
I20260807 15:54:27.190029 7740 ts_tablet_manager.cc:1428] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: Time spent starting tablet: real 0.033s user 0.017s sys 0.004s
I20260807 15:54:27.193567 7738 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: No bootstrap required, opened a new log
I20260807 15:54:27.194092 7738 ts_tablet_manager.cc:1397] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.014s
I20260807 15:54:27.197227 7738 raft_consensus.cc:348] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.197973 7738 raft_consensus.cc:374] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.198287 7738 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.199026 7738 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.202745 7737 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: Bootstrap starting.
I20260807 15:54:27.209893 7737 tablet_bootstrap.cc:654] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.211071 7738 ts_tablet_manager.cc:1428] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Time spent starting tablet: real 0.017s user 0.004s sys 0.003s
I20260807 15:54:27.213830 7748 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.214347 7748 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.218178 7746 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.218720 7738 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: Bootstrap starting.
I20260807 15:54:27.217168 7748 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.218925 7746 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.241710 7740 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: Bootstrap starting.
I20260807 15:54:27.246701 7745 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.247447 7745 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.256031 7745 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.282120 7746 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.286974 7738 tablet_bootstrap.cc:654] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.339362 7386 catalog_manager.cc:1259] Loaded cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:27.370262 7386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: loading cluster ID for follower catalog manager: success
I20260807 15:54:27.374689 7411 catalog_manager.cc:1259] Loaded cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:27.375005 7411 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: loading cluster ID for follower catalog manager: success
I20260807 15:54:27.385965 7745 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.390166 7386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: acquiring CA information for follower catalog manager: success
I20260807 15:54:27.389667 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:27.386526 7745 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.392001 7737 tablet_bootstrap.cc:492] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: No bootstrap required, opened a new log
I20260807 15:54:27.395135 7745 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
W20260807 15:54:27.396147 7442 leader_election.cc:341] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Illegal state: must be running to vote when last-logged opid is not known
I20260807 15:54:27.396970 7737 ts_tablet_manager.cc:1397] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: Time spent bootstrapping tablet: real 0.194s user 0.039s sys 0.066s
I20260807 15:54:27.404381 7740 tablet_bootstrap.cc:654] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.407722 7737 raft_consensus.cc:348] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.409920 7737 raft_consensus.cc:374] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.410231 7737 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.415073 7737 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.417376 7738 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: No bootstrap required, opened a new log
I20260807 15:54:27.417975 7738 ts_tablet_manager.cc:1397] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: Time spent bootstrapping tablet: real 0.199s user 0.010s sys 0.017s
I20260807 15:54:27.420857 7738 raft_consensus.cc:348] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.423528 7738 raft_consensus.cc:374] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.423972 7738 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.423187 7737 ts_tablet_manager.cc:1428] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: Time spent starting tablet: real 0.026s user 0.000s sys 0.008s
I20260807 15:54:27.424883 7411 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: acquiring CA information for follower catalog manager: success
I20260807 15:54:27.433166 7737 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: Bootstrap starting.
I20260807 15:54:27.441603 7737 tablet_bootstrap.cc:654] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.433804 7738 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.443764 7738 ts_tablet_manager.cc:1428] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260807 15:54:27.450352 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
W20260807 15:54:27.460114 7637 leader_election.cc:341] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Illegal state: must be running to vote when last-logged opid is not known
I20260807 15:54:27.472405 7386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:54:27.491792 7411 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:54:27.478268 7746 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.499534 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.500550 7484 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 0.
I20260807 15:54:27.487036 7745 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.506305 7745 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.509516 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.495641 7746 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.517226 7746 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.513228 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:27.518764 7592 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 0.
I20260807 15:54:27.508822 7745 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.524794 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.525862 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 0.
I20260807 15:54:27.527758 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:27.528992 7746 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:27.529511 7746 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.529927 7746 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Advancing to term 1
W20260807 15:54:27.534471 7542 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 62813.6us
I20260807 15:54:27.542579 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.542644 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.544225 7487 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 0.
I20260807 15:54:27.544343 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 0.
I20260807 15:54:27.546430 7441 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters: f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.547621 7748 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:27.548230 7748 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.546451 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.550239 7748 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.552244 7740 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: No bootstrap required, opened a new log
W20260807 15:54:27.554580 7542 leader_election.cc:341] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Illegal state: must be running to vote when last-logged opid is not known
I20260807 15:54:27.550480 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.555670 7737 tablet_bootstrap.cc:492] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: No bootstrap required, opened a new log
I20260807 15:54:27.555888 7686 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 0.
I20260807 15:54:27.556159 7737 ts_tablet_manager.cc:1397] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: Time spent bootstrapping tablet: real 0.123s user 0.019s sys 0.003s
W20260807 15:54:27.550573 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 56376.2us
I20260807 15:54:27.557125 7539 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:27.558933 7748 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.559782 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.560411 7740 ts_tablet_manager.cc:1397] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: Time spent bootstrapping tablet: real 0.323s user 0.019s sys 0.018s
I20260807 15:54:27.561693 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:27.564146 7745 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:27.564528 7745 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.564656 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:27.566013 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:27.566553 7748 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 1 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.564934 7745 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.566685 7746 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.568552 7591 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 1: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:27.568759 7740 raft_consensus.cc:348] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.569778 7740 raft_consensus.cc:374] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.570135 7740 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.570906 7740 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
W20260807 15:54:27.571640 7637 leader_election.cc:341] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Illegal state: must be running to vote when last-logged opid is not known
I20260807 15:54:27.573180 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.573846 7484 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 0.
I20260807 15:54:27.574786 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:27.575872 7487 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 1: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:27.578073 7746 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.578218 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters: f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.578910 7746 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:27.559296 7737 raft_consensus.cc:348] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.562258 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:27.579378 7746 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.579802 7737 raft_consensus.cc:374] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.579905 7746 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.581229 7745 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.582453 7686 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 1: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.580238 7737 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.586015 7737 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.586565 7441 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.589408 7737 ts_tablet_manager.cc:1428] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: Time spent starting tablet: real 0.033s user 0.006s sys 0.003s
I20260807 15:54:27.589355 7746 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.591163 7592 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 1: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:27.591377 7745 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.591988 7745 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:54:27.592345 7745 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.592693 7745 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.591064 7772 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 15:54:27.595430 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:27.596133 7740 ts_tablet_manager.cc:1428] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: Time spent starting tablet: real 0.032s user 0.003s sys 0.004s
W20260807 15:54:27.597193 7542 leader_election.cc:341] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Illegal state: must be running to vote when last-logged opid is not known
I20260807 15:54:27.598397 7746 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.600188 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.600313 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:27.602509 7592 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 1: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:27.602878 7771 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.604624 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:27.605422 7487 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 1: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:27.608942 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.612934 7740 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: Bootstrap starting.
W20260807 15:54:27.620786 7539 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.3:34107, user_credentials={real_user=slave}} blocked reactor thread for 62905us
I20260807 15:54:27.624466 7745 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.626156 7745 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.627211 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:27.627244 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:27.627934 7487 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.628691 7771 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 15:54:27.629577 7769 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.630409 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:27.630298 7769 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.629571 7773 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 15:54:27.631669 7592 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.631795 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:27.632687 7686 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 1: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.636027 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:27.636808 7746 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 15:54:27.638463 7487 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 1.
I20260807 15:54:27.639294 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.638480 7769 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.639433 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.640651 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:27.641263 7686 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 1: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.641021 7685 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 1.
I20260807 15:54:27.641785 7773 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:27.641021 7592 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 1.
I20260807 15:54:27.642310 7773 raft_consensus.cc:686] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 LEADER]: Becoming Leader. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:27.643525 7740 tablet_bootstrap.cc:654] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:54:27.643270 7773 consensus_queue.cc:237] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.640187 7484 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 1: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.645735 7539 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:27.646689 7769 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 15:54:27.647156 7539 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:27.648172 7769 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:54:27.650996 7769 raft_consensus.cc:686] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:27.651739 7769 consensus_queue.cc:237] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.656430 7740 tablet_bootstrap.cc:492] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: No bootstrap required, opened a new log
I20260807 15:54:27.656946 7740 ts_tablet_manager.cc:1397] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: Time spent bootstrapping tablet: real 0.044s user 0.012s sys 0.000s
I20260807 15:54:27.659055 7221 catalog_manager.cc:5495] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 reported cstate change: term changed from 0 to 1, leader changed from <none> to d77851634f654ce5bfce79a4dd797055 (127.5.45.3). New cstate: current_term: 1 leader_uuid: "d77851634f654ce5bfce79a4dd797055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:27.660864 7740 raft_consensus.cc:348] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.661667 7740 raft_consensus.cc:374] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:54:27.661972 7740 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Initialized, Role: FOLLOWER
I20260807 15:54:27.662717 7740 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.664954 7740 ts_tablet_manager.cc:1428] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 15:54:27.667770 7222 catalog_manager.cc:5495] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 1 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:27.734886 7769 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:27.743506 7484 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:54:27.747700 7773 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 15:54:27.755532 7746 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 15:54:27.759332 7773 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:27.821827 7772 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.822580 7772 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.835563 7771 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.836349 7771 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.839005 7771 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.845502 7772 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.840772 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:27.853868 7592 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 1.
I20260807 15:54:27.847185 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.855186 7685 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 1.
I20260807 15:54:27.848556 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.856389 7487 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 1.
I20260807 15:54:27.850448 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:27.855796 7769 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:27.858494 7769 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.859036 7441 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:27.859529 7591 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 1.
I20260807 15:54:27.861431 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:27.862545 7769 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.861660 7772 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:54:27.863808 7771 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:54:27.864184 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:27.865541 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:27.867741 7771 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.868201 7771 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:27.866278 7772 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:27.869959 7772 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:27.877566 7771 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.877152 7772 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.878800 7686 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 2: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.879037 7487 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 2: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:27.881101 7772 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:27.880995 7542 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:27.882535 7793 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:27.882925 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:27.883882 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:27.884320 7771 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:27.884645 7686 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 2: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:27.885075 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:27.885792 7487 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 2: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:27.887814 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:27.889017 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:27.889887 7771 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 15:54:27.891336 7793 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:54:27.895802 7591 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 2.
I20260807 15:54:27.897284 7442 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:27.898204 7772 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:54:27.898648 7772 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 LEADER]: Becoming Leader. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:27.899616 7772 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:27.907691 7222 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed reported cstate change: term changed from 0 to 2, leader changed from <none> to f07486fcec2343278bcc737786ff6fed (127.5.45.1). New cstate: current_term: 2 leader_uuid: "f07486fcec2343278bcc737786ff6fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:27.987819 7591 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 15:54:27.989696 7772 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:54:28.003919 7686 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 15:54:28.006361 7772 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 15:54:28.221598 7745 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:28.222316 7745 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.226989 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:28.228525 7771 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:28.230680 7745 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:28.223732 7793 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:28.229189 7771 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.232174 7793 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.233500 7769 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:28.233987 7769 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.267912 7769 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:28.269917 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:28.269924 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:28.271780 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 2.
I20260807 15:54:28.286435 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:28.267912 7771 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:28.353605 7793 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:28.354548 7793 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:28.355108 7793 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:28.356820 7793 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:28.357182 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:28.389149 7803 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:28.389815 7803 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:28.391983 7803 leader_election.cc:288] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549)
I20260807 15:54:28.400517 7793 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.402212 7802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:28.403594 7802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:28.407444 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:28.408149 7685 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:28.407485 7802 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
W20260807 15:54:28.469645 7637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.45.1:42027 timed out after 0.210s (SENT) [suppressed 17 similar messages]
I20260807 15:54:28.416781 7793 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:28.469861 7685 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 3.
I20260807 15:54:28.550029 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:28.652292 7769 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 15:54:28.661499 7542 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 109333us
W20260807 15:54:28.677436 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 205282us
I20260807 15:54:28.685904 7769 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:28.687040 7769 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.690582 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7" is_pre_election: true
I20260807 15:54:28.691391 7144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3acfcc965f048a4908d9687c07fd330 in term 1.
I20260807 15:54:28.694319 7283 leader_election.cc:302] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, a3acfcc965f048a4908d9687c07fd330; no voters:
I20260807 15:54:28.695706 7803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:54:28.696146 7803 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Starting leader election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:28.696538 7803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.705242 7483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.701937 7803 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:28.706737 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.706877 7483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:28.707460 7480 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 1.
I20260807 15:54:28.708752 7803 leader_election.cc:288] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 2 election: Requested vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549)
I20260807 15:54:28.709237 7542 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:28.710035 7769 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:54:28.710421 7769 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:28.710661 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.712677 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7"
I20260807 15:54:28.713163 7144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.710912 7769 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.718211 7144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3acfcc965f048a4908d9687c07fd330 in term 2.
W20260807 15:54:28.719269 7145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52398 (request call id 1) took 316 ms (client timeout 242 ms). Trace:
I20260807 15:54:28.719730 7283 leader_election.cc:302] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, a3acfcc965f048a4908d9687c07fd330; no voters:
I20260807 15:54:28.720876 7803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:54:28.721572 7147 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Rejecting Update request from peer df6a37abab2d49529a2a2ca42d10c7cf for earlier term 1. Current term is 2. Ops: []
I20260807 15:54:28.724900 7769 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
W20260807 15:54:28.719756 7145 rpcz_store.cc:269] 0807 15:54:28.402841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:28.718802 (+315961us) service_pool.cc:214] Skipping call since client already timed out
0807 15:54:28.719047 (+ 245us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260807 15:54:28.727064 7769 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:28.727941 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:28.728143 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:28.728147 7803 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 LEADER]: Becoming Leader. State: Replica: a3acfcc965f048a4908d9687c07fd330, State: Running, Role: LEADER
I20260807 15:54:28.728583 7481 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.730489 7482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:28.730767 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.733847 7803 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:28.736413 7481 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 2.
I20260807 15:54:28.737731 7542 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:28.738554 7769 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:54:28.739112 7769 raft_consensus.cc:686] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:28.740523 7769 consensus_queue.cc:237] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.745035 7481 raft_consensus.cc:1226] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Rejecting Update request from peer d77851634f654ce5bfce79a4dd797055 for earlier term 1. Current term is 2. Ops: []
I20260807 15:54:28.746162 7482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.746657 7771 consensus_queue.cc:1046] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:28.747841 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.749559 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.752565 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.754397 7811 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3acfcc965f048a4908d9687c07fd330. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.755265 7811 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:28.755884 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:28.757186 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:28.767145 7317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:28.771741 7591 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 2. Current term is 3. Ops: []
I20260807 15:54:28.790742 7316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:28.802263 7771 raft_consensus.cc:3032] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:28.802992 7771 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:28.803728 7685 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 2. Current term is 3. Ops: []
I20260807 15:54:28.808717 7771 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.811913 7771 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.835173 7818 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 15:54:28.849082 7145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Refusing update from remote peer a3acfcc965f048a4908d9687c07fd330: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260807 15:54:28.850770 7811 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 15:54:28.859894 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:28.861835 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:28.865532 7481 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:54:28.870142 7810 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260807 15:54:28.807188 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 50559.1us
I20260807 15:54:28.876745 7797 consensus_queue.cc:1046] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 15:54:28.876868 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:28.877532 7797 consensus_queue.cc:1225] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer f847539a423f4000a7839152f3a1fa99 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 15:54:28.878278 7814 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:54:28.873178 7824 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:54:28.883908 7811 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.884802 7811 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:28.887043 7824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.887920 7824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:28.894668 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:28.894928 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf"
I20260807 15:54:28.895666 7239 raft_consensus.cc:3032] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:28.896031 7239 raft_consensus.cc:729] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df6a37abab2d49529a2a2ca42d10c7cf, State: Running, Role: LEADER
I20260807 15:54:28.896737 7239 consensus_queue.cc:260] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:28.897696 7239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:28.902107 7484 raft_consensus.cc:3032] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 LEADER]: Stepping down as leader of term 2
I20260807 15:54:28.902503 7484 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:28.903281 7484 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.904506 7484 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:28.907363 7239 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Refusing update from remote peer a3acfcc965f048a4908d9687c07fd330: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260807 15:54:28.911376 7484 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 15:54:28.912475 7803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 15:54:28.913563 7769 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:28.920941 7831 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.922065 7831 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:28.934226 7816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:28.949005 7686 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:54:28.948725 7802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3acfcc965f048a4908d9687c07fd330. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.954370 7802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:28.973459 7683 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 15:54:28.977509 7723 master_proxy_rpc.cc:203] Re-attempting IsCreateTableDone request to leader Master (127.5.45.61:36549)
I20260807 15:54:28.975171 7769 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260807 15:54:28.983474 7771 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:28.984036 7771 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:28.988062 7771 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:28.988209 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:28.990011 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:28.991397 7793 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:54:28.991421 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:28.997378 7828 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3acfcc965f048a4908d9687c07fd330. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:28.998201 7828 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:28.999958 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a3acfcc965f048a4908d9687c07fd330" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:29.000711 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:29.014021 7816 catalog_manager.cc:669] Loaded metadata for table test-table [id=549be6f3994e45de82118121367e9e71]
I20260807 15:54:29.022219 7814 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 15:54:29.027581 7816 tablet_loader.cc:96] loaded metadata for tablet 21d3feeebcad478aa30ccdde78044663 (table test-table [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:29.035017 7416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:54:29.035907 7816 tablet_loader.cc:96] loaded metadata for tablet 7d2b29f18f024df199716835932639a1 (table test-table [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:29.037251 7816 tablet_loader.cc:96] loaded metadata for tablet fcb581517aa04c7dbeba2ba716118eed (table test-table [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:29.038245 7816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:29.041489 7816 catalog_manager.cc:1259] Loaded cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:29.041757 7816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:29.045601 7816 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:29.049433 7816 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Loaded TSK: 0
I20260807 15:54:29.050468 7816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:54:29.089785 7304 catalog_manager.cc:5495] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 1 to 2, leader changed from d77851634f654ce5bfce79a4dd797055 (127.5.45.3) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 2 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:29.091872 7304 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 2 to 3, leader changed from f07486fcec2343278bcc737786ff6fed (127.5.45.1) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 3 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:29.128317 7619 heartbeater.cc:503] Master 127.5.45.60:42447 was elected leader, sending a full tablet report...
W20260807 15:54:29.163590 7458 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59426: Illegal state: replica f07486fcec2343278bcc737786ff6fed is not leader of this config: current role FOLLOWER
W20260807 15:54:29.163975 7662 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60662: Illegal state: replica d77851634f654ce5bfce79a4dd797055 is not leader of this config: current role FOLLOWER
W20260807 15:54:29.169248 7458 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59426: Illegal state: replica f07486fcec2343278bcc737786ff6fed is not leader of this config: current role FOLLOWER
I20260807 15:54:29.180794 7519 heartbeater.cc:503] Master 127.5.45.60:42447 was elected leader, sending a full tablet report...
I20260807 15:54:29.184665 7713 heartbeater.cc:503] Master 127.5.45.60:42447 was elected leader, sending a full tablet report...
I20260807 15:54:29.490873 7591 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed"
dest_uuid: "f847539a423f4000a7839152f3a1fa99"
from {username='slave'} at 127.0.0.1:37800
I20260807 15:54:29.492586 7591 raft_consensus.cc:580] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 LEADER]: Received request to step down
I20260807 15:54:29.493250 7591 raft_consensus.cc:3032] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 15:54:29.493660 7591 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:29.494732 7591 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:29.496484 7591 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:29.686842 7889 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:29.687402 7887 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:29.687791 7889 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:29.688190 7887 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:29.690588 7887 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:29.691615 7889 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:29.692605 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:29.693593 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:29.693742 7686 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 2.
I20260807 15:54:29.693868 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:29.694410 7592 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 3.
I20260807 15:54:29.695309 7441 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:29.696568 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:29.697582 7481 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 2.
I20260807 15:54:29.698976 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters: f847539a423f4000a7839152f3a1fa99
I20260807 15:54:29.707222 7889 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:29.707866 7889 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:29.714049 7889 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:29.714143 7887 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:29.719647 7887 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:29.766680 7887 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:29.731050 7889 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:29.776391 7889 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:31.403226 7890 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:31.404078 7890 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
W20260807 15:54:31.408924 7887 consensus_meta.cc:287] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Time spent flushing consensus metadata: real 1.641s user 0.011s sys 0.000s
I20260807 15:54:31.409621 7887 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:31.453408 7890 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:31.454332 7890 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:31.455016 7890 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
W20260807 15:54:31.878167 5306 debug-util.cc:398] Leaking SignalData structure 0x7b08000287c0 after lost signal to thread 7635
W20260807 15:54:31.913174 7639 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.032s user 0.000s sys 0.006s
W20260807 15:54:31.914517 7639 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.102s user 0.001s sys 0.006s
W20260807 15:54:31.915355 7635 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.807s user 0.070s sys 0.120s
W20260807 15:54:31.915788 7635 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.948s user 0.071s sys 0.120s
I20260807 15:54:31.920233 7890 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:31.920306 7887 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:31.988634 7894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3acfcc965f048a4908d9687c07fd330)
I20260807 15:54:31.989441 7894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:31.992445 7894 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:32.024760 7222 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 27) took 2333 ms. Trace:
I20260807 15:54:32.025278 7222 rpcz_store.cc:276] 0807 15:54:29.691357 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:32.023082 (+2331725us) service_pool.cc:227] Handling call
0807 15:54:32.024562 (+ 1480us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260807 15:54:32.032917 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:32.042135 7896 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:32.042943 7896 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.043342 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:32.045459 7896 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.045765 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:32.046422 7487 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 3.
I20260807 15:54:32.047616 7484 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 3: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:32.047443 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:32.048398 7887 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 15:54:32.048775 7887 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:32.049182 7887 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:32.050191 7316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:32.052850 7111 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:32.053755 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:32.054363 7480 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 1.
I20260807 15:54:32.055315 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:32.056216 7890 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:54:32.056583 7890 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:32.056941 7890 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:32.057063 7894 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 15:54:32.059211 7887 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.062909 7887 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:32.063365 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:32.064082 7890 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.064476 7480 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:32.062927 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:32.066015 7890 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:32.067555 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:32.068243 7481 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:32.069064 7221 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56944 (request call id 26) took 2377 ms. Trace:
I20260807 15:54:32.069396 7221 rpcz_store.cc:276] 0807 15:54:29.691746 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:32.067851 (+2376105us) service_pool.cc:227] Handling call
0807 15:54:32.068938 (+ 1087us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260807 15:54:32.072710 7480 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 4.
W20260807 15:54:32.074707 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080019e940 after lost signal to thread 5304
W20260807 15:54:32.076244 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080015ec80 after lost signal to thread 7174
I20260807 15:54:32.076325 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.077086 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
W20260807 15:54:32.077713 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080015d820 after lost signal to thread 7267
I20260807 15:54:32.078430 7890 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 15:54:32.081425 7686 raft_consensus.cc:1226] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 1. Current term is 2. Ops: []
I20260807 15:54:32.084693 7587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.086102 7592 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 3.
I20260807 15:54:32.086144 7589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.087942 7480 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:32.087937 7442 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:32.090647 7589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.092442 7890 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 LEADER]: Becoming Leader. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:32.093827 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.092617 7895 consensus_queue.cc:1046] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 15:54:32.095690 7896 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:54:32.076498 7481 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 2.
I20260807 15:54:32.098305 7484 raft_consensus.cc:1226] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 1. Current term is 2. Ops: []
W20260807 15:54:32.098793 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080023cfe0 after lost signal to thread 7776
I20260807 15:54:32.097441 7591 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663"
dest_uuid: "f847539a423f4000a7839152f3a1fa99"
from {username='slave'} at 127.0.0.1:37800
I20260807 15:54:32.095602 7588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.099493 7591 raft_consensus.cc:580] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Received request to step down
I20260807 15:54:32.099869 7591 raft_consensus.cc:3032] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 15:54:32.100165 7591 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
W20260807 15:54:32.099211 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800233e20 after lost signal to thread 7790
I20260807 15:54:32.101040 7591 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.095573 7890 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.107755 7896 raft_consensus.cc:686] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 LEADER]: Becoming Leader. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:32.108966 7896 consensus_queue.cc:237] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.112839 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:32.113792 7683 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 2: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:32.114531 7895 raft_consensus.cc:3032] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 15:54:32.114946 7895 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:32.115835 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.118897 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:32.119801 7686 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 3: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:32.114580 7898 consensus_queue.cc:1046] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260807 15:54:32.115867 7895 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.119488 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.130365 7895 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:54:32.154518 7683 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 LEADER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:32.157068 7590 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 2.
W20260807 15:54:32.157073 7542 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 60768us
I20260807 15:54:32.175323 7906 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3acfcc965f048a4908d9687c07fd330)
I20260807 15:54:32.175921 7906 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:32.178066 7906 leader_election.cc:288] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:32.179401 7316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:32.182353 7145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7" is_pre_election: true
I20260807 15:54:32.184845 7199 leader_election.cc:302] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df6a37abab2d49529a2a2ca42d10c7cf; no voters: 4dc450c6e3db439994750939886f3aa7, a3acfcc965f048a4908d9687c07fd330
I20260807 15:54:32.185745 7906 raft_consensus.cc:2726] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 15:54:32.204416 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:32.205369 7896 raft_consensus.cc:2741] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260807 15:54:32.214105 7686 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 15:54:32.215957 7896 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:54:32.228349 7590 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 15:54:32.230144 7896 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 15:54:32.229930 7305 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 reported cstate change: term changed from 3 to 4, leader changed from f847539a423f4000a7839152f3a1fa99 (127.5.45.2) to d77851634f654ce5bfce79a4dd797055 (127.5.45.3). New cstate: current_term: 4 leader_uuid: "d77851634f654ce5bfce79a4dd797055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:32.231016 7304 catalog_manager.cc:5495] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed reported cstate change: term changed from 2 to 3, leader changed from f847539a423f4000a7839152f3a1fa99 (127.5.45.2) to f07486fcec2343278bcc737786ff6fed (127.5.45.1). New cstate: current_term: 3 leader_uuid: "f07486fcec2343278bcc737786ff6fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
W20260807 15:54:32.239228 7637 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.294s (SENT)
I20260807 15:54:32.246057 7481 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 15:54:32.254165 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters: f847539a423f4000a7839152f3a1fa99
I20260807 15:54:32.255229 7890 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:54:32.255723 7890 raft_consensus.cc:686] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 LEADER]: Becoming Leader. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:32.256529 7890 consensus_queue.cc:237] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.257297 7588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.258044 7588 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 2.
I20260807 15:54:32.259148 7589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.261106 7887 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:54:32.260166 7591 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:32.265239 7586 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 4.
I20260807 15:54:32.268090 7592 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 15:54:32.283819 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:32.296975 7891 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:54:32.348218 7887 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 15:54:32.392750 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.45.2:37981, user_credentials={real_user=slave}} blocked reactor thread for 68111.3us
I20260807 15:54:32.403294 7592 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 15:54:32.404943 7481 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 15:54:32.413211 7887 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 15:54:32.424214 7921 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260807 15:54:32.509567 7302 catalog_manager.cc:5495] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 1 to 2, leader changed from f847539a423f4000a7839152f3a1fa99 (127.5.45.2) to d77851634f654ce5bfce79a4dd797055 (127.5.45.3). New cstate: current_term: 2 leader_uuid: "d77851634f654ce5bfce79a4dd797055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } } }
I20260807 15:54:32.570652 7683 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7d2b29f18f024df199716835932639a1"
dest_uuid: "d77851634f654ce5bfce79a4dd797055"
from {username='slave'} at 127.0.0.1:40692
I20260807 15:54:32.571281 7683 raft_consensus.cc:580] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 LEADER]: Received request to step down
I20260807 15:54:32.571574 7683 raft_consensus.cc:3032] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 15:54:32.571820 7683 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:32.572552 7683 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.573664 7683 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:32.715497 7898 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.716073 7898 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.718322 7898 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.719559 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:32.719604 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:32.720395 7683 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 3.
I20260807 15:54:32.721532 7539 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:32.722273 7898 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:32.722656 7898 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.722970 7898 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:32.723278 7896 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.723845 7896 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.730000 7896 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.729929 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:32.731091 7683 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 3.
I20260807 15:54:32.729914 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.730186 7898 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.732858 7586 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 3: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:32.733282 7441 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:32.736109 7898 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.737998 7900 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:32.738287 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:32.739809 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:32.738425 7900 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Starting leader election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.741051 7900 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:32.748142 7900 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.749259 7481 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 3: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:32.752146 7900 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.752324 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:32.752373 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.754129 7683 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 3.
I20260807 15:54:32.754539 7586 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 3: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:32.755218 7539 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:32.756302 7898 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:54:32.756376 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:32.757244 7896 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 15:54:32.758363 7898 raft_consensus.cc:686] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:32.759289 7898 consensus_queue.cc:237] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.773240 7304 catalog_manager.cc:5495] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 2 to 3, leader changed from d77851634f654ce5bfce79a4dd797055 (127.5.45.3) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 3 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:32.778168 7683 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663"
dest_uuid: "d77851634f654ce5bfce79a4dd797055"
from {username='slave'} at 127.0.0.1:40692
I20260807 15:54:32.778816 7683 raft_consensus.cc:580] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 LEADER]: Received request to step down
I20260807 15:54:32.779222 7683 raft_consensus.cc:3032] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 15:54:32.779529 7683 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:32.780689 7683 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.782795 7683 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:32.864287 7686 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 15:54:32.865721 7898 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:54:32.868773 7481 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 15:54:32.870046 7939 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.870536 7898 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:54:32.870615 7939 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.874122 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:32.878561 7896 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.879063 7896 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.883250 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:32.884647 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:32.884912 7586 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 4.
I20260807 15:54:32.886202 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 5.
I20260807 15:54:32.887281 7441 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:32.888271 7900 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 15:54:32.888656 7900 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting leader election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:32.889034 7900 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:32.881299 7896 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.913964 7900 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.918725 7939 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.920450 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:32.921310 7586 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:32.920913 7900 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:32.926524 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:32.927284 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 5.
I20260807 15:54:32.928742 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:32.934863 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:32.951792 7686 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 5.
I20260807 15:54:32.953752 7441 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:32.954938 7900 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader election won for term 5
I20260807 15:54:32.955477 7900 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 LEADER]: Becoming Leader. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:32.956360 7900 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:32.942068 7586 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 5.
I20260807 15:54:32.959782 7945 raft_consensus.cc:2741] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260807 15:54:32.965281 7305 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed reported cstate change: term changed from 4 to 5, leader changed from d77851634f654ce5bfce79a4dd797055 (127.5.45.3) to f07486fcec2343278bcc737786ff6fed (127.5.45.1). New cstate: current_term: 5 leader_uuid: "f07486fcec2343278bcc737786ff6fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:33.007091 7305 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49716:
table { table_id: "549be6f3994e45de82118121367e9e71" table_name: "test-table" } new_table_name: "NewTableName"
I20260807 15:54:33.040712 5300 master.cc:537] Master@127.5.45.60:42447 shutting down...
I20260807 15:54:33.056417 7686 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 15:54:33.066455 7586 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 15:54:33.068102 7946 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 15:54:33.069546 7896 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 15:54:33.149214 7615 debug-util.cc:398] Leaking SignalData structure 0x7b08001d9060 after lost signal to thread 5304
W20260807 15:54:33.150475 7615 debug-util.cc:398] Leaking SignalData structure 0x7b08001d97c0 after lost signal to thread 7174
W20260807 15:54:33.151563 7615 debug-util.cc:398] Leaking SignalData structure 0x7b08001d9e80 after lost signal to thread 7267
W20260807 15:54:33.152516 7615 debug-util.cc:398] Leaking SignalData structure 0x7b080023a7c0 after lost signal to thread 7349
I20260807 15:54:33.150434 7958 tablet.cc:1722] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.167758 7960 tablet.cc:1722] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.170845 7954 tablet.cc:1722] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.220916 7958 tablet.cc:1722] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260807 15:54:33.242661 7442 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.45.2:37981, user_credentials={real_user=slave}} blocked reactor thread for 106817us
I20260807 15:54:33.258499 7961 tablet.cc:1722] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.273902 7955 tablet.cc:1722] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.296974 7964 tablet.cc:1722] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.304360 7962 tablet.cc:1722] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.335451 7945 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:33.336019 7945 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:33.338887 7481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:33.341308 7945 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:33.342598 7955 tablet.cc:1722] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260807 15:54:33.346896 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:33.350221 7539 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:33.354297 7944 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:33.370584 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 LEADER]: Raft consensus shutting down.
I20260807 15:54:33.371718 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:54:33.372210 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: stopping tablet replica
I20260807 15:54:33.414295 5300 master.cc:552] Master@127.5.45.60:42447 shutdown complete.
I20260807 15:54:33.506573 7972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3acfcc965f048a4908d9687c07fd330)
I20260807 15:54:33.507231 7972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:33.509619 7972 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:33.510629 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
W20260807 15:54:33.516703 7111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260807 15:54:33.532256 7111 leader_election.cc:334] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447): Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:33.532011 7974 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3acfcc965f048a4908d9687c07fd330)
I20260807 15:54:33.532874 7111 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:33.534039 7972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 15:54:33.533712 7974 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:33.537843 7145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7" is_pre_election: true
I20260807 15:54:33.538587 7145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 2.
I20260807 15:54:33.540138 7199 leader_election.cc:302] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf; no voters:
I20260807 15:54:33.543054 7974 leader_election.cc:288] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:33.543696 7974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:54:33.544071 7974 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a3acfcc965f048a4908d9687c07fd330)
I20260807 15:54:33.544467 7974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:33.571075 7974 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:33.575134 7145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7"
I20260807 15:54:33.575856 7145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:33.579943 7974 leader_election.cc:288] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 election: Requested vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:33.584030 7145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df6a37abab2d49529a2a2ca42d10c7cf in term 3.
I20260807 15:54:33.585278 7199 leader_election.cc:302] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf; no voters:
I20260807 15:54:33.586308 7974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:54:33.587121 7974 raft_consensus.cc:686] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 3 LEADER]: Becoming Leader. State: Replica: df6a37abab2d49529a2a2ca42d10c7cf, State: Running, Role: LEADER
W20260807 15:54:33.588617 7195 leader_election.cc:334] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447): Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:33.588178 7974 consensus_queue.cc:237] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
W20260807 15:54:33.593401 7195 leader_election.cc:334] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447): Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111)
I20260807 15:54:33.597576 7977 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:33.598552 7977 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:33.606395 7983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:54:33.615562 7983 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=549be6f3994e45de82118121367e9e71]
I20260807 15:54:33.623881 7983 tablet_loader.cc:96] loaded metadata for tablet 21d3feeebcad478aa30ccdde78044663 (table NewTableName [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:33.626281 7983 tablet_loader.cc:96] loaded metadata for tablet 7d2b29f18f024df199716835932639a1 (table NewTableName [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:33.627700 7983 tablet_loader.cc:96] loaded metadata for tablet fcb581517aa04c7dbeba2ba716118eed (table NewTableName [id=549be6f3994e45de82118121367e9e71])
I20260807 15:54:33.629487 7983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:54:33.634986 7983 catalog_manager.cc:1259] Loaded cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:33.635336 7983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:54:33.638942 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:54:33.642253 7983 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:54:33.647819 7983 catalog_manager.cc:5879] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: Loaded TSK: 0
I20260807 15:54:33.649627 7983 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:54:33.657841 7984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:33.665138 7986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:54:33.669374 7989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:54:33.675647 5300 server_base.cc:1034] running on GCE node
I20260807 15:54:33.680892 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:54:33.681286 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:54:33.681661 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118073681641 us; error 0 us; skew 500 ppm
I20260807 15:54:33.682504 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:54:33.690285 5300 webserver.cc:460] Webserver started at http://127.5.45.60:43949/ using document root <none> and password file <none>
I20260807 15:54:33.691012 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:54:33.691246 5300 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 15:54:33.713209 5300 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.007s sys 0.005s
I20260807 15:54:33.716615 7145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Refusing update from remote peer df6a37abab2d49529a2a2ca42d10c7cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 17. Preceding OpId from leader: term: 3 index: 18. (index mismatch)
I20260807 15:54:33.719151 7977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260807 15:54:33.725656 7994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:54:33.728161 5300 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.007s
I20260807 15:54:33.728677 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
uuid: "a3acfcc965f048a4908d9687c07fd330"
format_stamp: "Formatted at 2026-08-07 15:54:26 on dist-test-slave-bw88"
I20260807 15:54:33.729274 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260807 15:54:33.738929 7195 consensus_peers.cc:487] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf -> Peer a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447): Couldn't send request to peer a3acfcc965f048a4908d9687c07fd330. Status: Network error: Client connection negotiation failed: client connection to 127.5.45.60:42447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:54:33.749084 7972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:33.751178 7972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:33.777396 7977 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:33.778350 7977 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:33.779606 7972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:33.780354 7972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:33.784232 7714 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:33.789328 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:54:33.791297 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:54:33.812604 7520 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:33.836652 7620 heartbeater.cc:503] Master 127.5.45.61:36549 was elected leader, sending a full tablet report...
I20260807 15:54:33.920958 7221 catalog_manager.cc:6040] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: NewTableName [id=549be6f3994e45de82118121367e9e71] alter complete (version 1)
I20260807 15:54:34.085407 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.60:42447
I20260807 15:54:34.085568 8062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.60:42447 every 8 connection(s)
I20260807 15:54:34.112550 8063 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:54:34.122222 8063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Bootstrap starting.
I20260807 15:54:34.185861 7619 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:34.202312 7519 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:34.240198 7713 heartbeater.cc:347] Connected to a master server at 127.5.45.60:42447
I20260807 15:54:34.266027 8063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=7 ignored=0} mutations{seen=14 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:54:34.267367 8063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: Bootstrap complete.
I20260807 15:54:34.271041 8063 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:34.271979 8063 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3acfcc965f048a4908d9687c07fd330, State: Initialized, Role: FOLLOWER
I20260807 15:54:34.272915 8063 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:34.285897 8063 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:54:34.286372 8034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:54:34.333614 8075 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6a37abab2d49529a2a2ca42d10c7cf. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:34.334585 8075 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:34.334213 8072 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:34.335022 8072 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:34.336048 8080 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:34.336829 8080 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:54:34.382416 8093 catalog_manager.cc:1259] Loaded cluster ID: 560b5faeb74442b5a135c60fd838724a
I20260807 15:54:34.382710 8093 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: loading cluster ID for follower catalog manager: success
I20260807 15:54:34.391047 8093 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: acquiring CA information for follower catalog manager: success
I20260807 15:54:34.397184 8093 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:54:34.399098 8020 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d77851634f654ce5bfce79a4dd797055" instance_seqno: 1786118066856312) as {username='slave'} at 127.0.0.1:49224; Asking this server to re-register.
I20260807 15:54:34.400969 7713 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:34.400810 8019 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f847539a423f4000a7839152f3a1fa99" instance_seqno: 1786118066677539) as {username='slave'} at 127.0.0.1:49234; Asking this server to re-register.
I20260807 15:54:34.402587 7619 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:34.404762 8019 ts_manager.cc:194] Registered new tserver with Master: d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:34.406483 8018 ts_manager.cc:194] Registered new tserver with Master: f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:34.409511 7977 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:54:34.410362 7977 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:54:34.411855 8018 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f07486fcec2343278bcc737786ff6fed" instance_seqno: 1786118066512539) as {username='slave'} at 127.0.0.1:49230; Asking this server to re-register.
I20260807 15:54:34.413704 7519 heartbeater.cc:464] Registering TS with master...
I20260807 15:54:34.418375 8020 ts_manager.cc:194] Registered new tserver with Master: f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027)
I20260807 15:54:38.508870 8113 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:38.509857 8113 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:38.512735 8113 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:38.538362 8113 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:38.518949 8112 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:38.547149 8112 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:38.560554 8112 leader_election.cc:288] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549)
I20260807 15:54:38.562470 8113 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:38.579962 8113 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:38.588521 8113 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:38.593747 8113 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:38.614790 8113 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
W20260807 15:54:38.784740 7637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.45.1:42027 timed out after 0.271s (SENT) [suppressed 6 similar messages]
W20260807 15:54:39.017372 7635 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.013s sys 0.034s
W20260807 15:54:39.017877 7635 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.565s user 0.013s sys 0.034s
W20260807 15:54:39.034482 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 249777us
W20260807 15:54:39.072566 7996 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.62:46661, user_credentials={real_user=slave}} blocked reactor thread for 47878.7us
W20260807 15:54:39.283623 7996 leader_election.cc:334] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661): Timed out: connection negotiation to 127.5.45.62:46661 for RPC RequestConsensusVote timed out after 0.210s (ON_OUTBOUND_QUEUE)
W20260807 15:54:39.307410 7637 leader_election.cc:334] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Timed out: RequestConsensusVote RPC to 127.5.45.1:42027 timed out after 0.271s (SENT)
W20260807 15:54:39.326455 7637 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.234s (SENT)
W20260807 15:54:39.327243 7637 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Timed out: RequestConsensusVote RPC to 127.5.45.1:42027 timed out after 0.234s (SENT)
I20260807 15:54:39.327688 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:39.328896 8110 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:39.347275 8122 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:39.348052 8122 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:39.352154 8122 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:39.360894 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:39.414501 7482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:39.415228 7480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:39.415925 7480 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 3.
W20260807 15:54:39.967306 7481 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59412 (request call id 35) took 607 ms (client timeout 271 ms). Trace:
W20260807 15:54:39.967846 7481 rpcz_store.cc:269] 0807 15:54:39.359561 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:39.967041 (+607480us) service_pool.cc:214] Skipping call since client already timed out
0807 15:54:39.967216 (+ 175us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260807 15:54:40.246084 7113 thread.cc:636] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.896s user 0.002s sys 0.009s
W20260807 15:54:40.246780 7113 thread.cc:602] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.897s user 0.003s sys 0.010s
W20260807 15:54:40.324239 7484 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59412 (request call id 36) took 964 ms (client timeout 291 ms). Trace:
W20260807 15:54:40.324862 7484 rpcz_store.cc:269] 0807 15:54:39.360000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:40.323981 (+963981us) service_pool.cc:214] Skipping call since client already timed out
0807 15:54:40.324138 (+ 157us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260807 15:54:40.346889 7997 leader_election.cc:334] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549): Timed out: connection negotiation to 127.5.45.61:36549 for RPC RequestConsensusVote timed out after 0.210s (ON_OUTBOUND_QUEUE)
I20260807 15:54:40.347550 7997 leader_election.cc:302] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3acfcc965f048a4908d9687c07fd330; no voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:40.359257 8128 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:40.366708 8126 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.367298 8126 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.369503 8126 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.370812 8126 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.371306 8126 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.373754 8126 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.374331 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.375252 7686 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 3.
I20260807 15:54:40.376418 7539 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:40.377868 8126 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 15:54:40.378300 8126 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.378342 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:40.378873 8126 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:40.384178 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
W20260807 15:54:40.387102 7487 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59412 (request call id 38) took 1026 ms (client timeout 271 ms). Trace:
W20260807 15:54:40.387544 7487 rpcz_store.cc:269] 0807 15:54:39.360711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 15:54:40.386916 (+1026205us) service_pool.cc:214] Skipping call since client already timed out
0807 15:54:40.387039 (+ 123us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260807 15:54:40.390398 8126 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.392688 8126 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.394635 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.395357 7683 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 5.
I20260807 15:54:40.436630 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:40.445182 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:40.446376 7683 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:40.443058 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:40.461606 8125 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 15:54:40.462196 8125 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.462697 8125 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Advancing to term 6
I20260807 15:54:40.462617 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.463982 7592 raft_consensus.cc:1226] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:40.482527 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.484618 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.485587 7592 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 4: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:40.485906 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.487947 8127 consensus_queue.cc:1046] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 15:54:40.490240 8131 raft_consensus.cc:3032] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 LEADER]: Stepping down as leader of term 3
W20260807 15:54:40.490585 7637 leader_election.cc:334] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.271s (SENT)
I20260807 15:54:40.490724 8131 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:40.491456 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.496156 7441 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.498798 8125 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.499681 7589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.500180 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.492689 8131 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.504997 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
W20260807 15:54:40.505631 7637 leader_election.cc:334] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.291s (SENT)
I20260807 15:54:40.506166 7592 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 4: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
W20260807 15:54:40.507000 7637 leader_election.cc:334] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Timed out: RequestConsensusVote RPC to 127.5.45.1:42027 timed out after 0.291s (SENT)
I20260807 15:54:40.505743 8131 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:40.507558 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.509852 8133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:40.512820 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.522166 7590 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 5. Current term is 6. Ops: []
I20260807 15:54:40.530630 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:40.510607 8133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:40.533205 8125 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.534693 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:40.539316 7539 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:40.539978 8131 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:40.541049 8126 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 15:54:40.557482 7683 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 4.
I20260807 15:54:40.560938 7685 raft_consensus.cc:1226] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:40.562619 8134 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:40.564535 8134 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:40.565995 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:40.570873 8133 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:40.573890 8127 consensus_queue.cc:1046] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 15:54:40.576633 7539 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:40.577994 8126 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 15:54:40.588886 7615 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe460 after lost signal to thread 5304
W20260807 15:54:40.590596 7615 debug-util.cc:398] Leaking SignalData structure 0x7b0800242ca0 after lost signal to thread 7174
W20260807 15:54:40.592057 7615 debug-util.cc:398] Leaking SignalData structure 0x7b0800251920 after lost signal to thread 7267
W20260807 15:54:40.594915 7615 debug-util.cc:398] Leaking SignalData structure 0x7b080026e7c0 after lost signal to thread 7518
W20260807 15:54:40.596345 7615 debug-util.cc:398] Leaking SignalData structure 0x7b080017c3a0 after lost signal to thread 7618
W20260807 15:54:40.598050 7615 debug-util.cc:398] Leaking SignalData structure 0x7b080016c480 after lost signal to thread 7712
W20260807 15:54:40.598347 7615 debug-util.cc:398] Leaking SignalData structure 0x7b080012fc60 after lost signal to thread 7714
W20260807 15:54:40.579957 7441 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.45.3:34107, user_credentials={real_user=slave}} blocked reactor thread for 55200us
W20260807 15:54:40.604203 7637 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.1:42027, user_credentials={real_user=slave}} blocked reactor thread for 97239.8us
I20260807 15:54:40.609663 7590 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 5. Current term is 6. Ops: []
I20260807 15:54:40.602324 8135 raft_consensus.cc:3032] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 LEADER]: Stepping down as leader of term 5
I20260807 15:54:40.612500 8138 consensus_queue.cc:1046] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260807 15:54:40.614975 8135 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:40.616026 8135 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.617712 8135 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Advancing to term 6
I20260807 15:54:40.620800 8126 raft_consensus.cc:686] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:40.625226 8126 consensus_queue.cc:237] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.648778 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:40.651332 7111 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:40.653113 8133 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:40.663686 7221 catalog_manager.cc:5495] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 3 to 4, leader changed from f07486fcec2343278bcc737786ff6fed (127.5.45.1) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 4 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:40.679805 8135 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:40.680485 8135 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.684396 8135 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.684715 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.684715 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.687225 7683 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 4.
I20260807 15:54:40.689000 7441 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:40.689545 8136 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.690048 8135 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 15:54:40.690459 8135 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:40.690083 8136 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.693871 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:40.694576 7487 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 6.
I20260807 15:54:40.695777 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:40.696663 8134 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 15:54:40.697042 8134 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:40.697431 8134 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Advancing to term 6
I20260807 15:54:40.690828 8135 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:40.706087 8136 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:40.707856 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.709168 7590 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 6: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:40.713105 7683 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 15:54:40.719197 8126 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:40.719408 8135 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.726262 7484 raft_consensus.cc:1226] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 4. Current term is 5. Ops: []
I20260807 15:54:40.727826 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:40.724280 8134 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.728976 8144 consensus_queue.cc:1046] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 15:54:40.730226 8144 raft_consensus.cc:3032] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 15:54:40.730595 8144 raft_consensus.cc:729] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:40.731419 8144 consensus_queue.cc:260] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.732658 8144 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:40.732862 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:40.741822 7484 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 6.
I20260807 15:54:40.761107 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:40.743911 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:40.767241 7590 raft_consensus.cc:2379] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 6: Already voted for candidate f847539a423f4000a7839152f3a1fa99 in this term.
I20260807 15:54:40.767975 7441 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.742657 8135 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 5 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.768904 8138 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 15:54:40.768397 8134 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:40.771456 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters: f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.772315 8134 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 15:54:40.773208 8134 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 LEADER]: Becoming Leader. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:40.775350 8134 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.824251 8144 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:40.824777 8144 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.827209 8144 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.828799 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.831733 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:40.832412 7484 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 6.
I20260807 15:54:40.833834 7542 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:40.835106 8144 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 15:54:40.835562 8144 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:40.840551 8144 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 6 FOLLOWER]: Advancing to term 7
I20260807 15:54:40.822767 7222 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 reported cstate change: term changed from 5 to 6, leader changed from f07486fcec2343278bcc737786ff6fed (127.5.45.1) to d77851634f654ce5bfce79a4dd797055 (127.5.45.3). New cstate: current_term: 6 leader_uuid: "d77851634f654ce5bfce79a4dd797055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:40.876482 8144 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.878970 8144 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 7 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.879345 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:40.879009 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:40.880821 7484 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 6 FOLLOWER]: Advancing to term 7
I20260807 15:54:40.887924 7484 raft_consensus.cc:2454] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 7.
I20260807 15:54:40.892488 7542 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:40.893636 8144 raft_consensus.cc:2781] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 15:54:40.894124 8144 raft_consensus.cc:686] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:40.895015 8144 consensus_queue.cc:237] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.901580 7487 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Rejecting Update request from peer d77851634f654ce5bfce79a4dd797055 for earlier term 6. Current term is 7. Ops: []
I20260807 15:54:40.904538 8136 consensus_queue.cc:1046] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:54:40.905519 8136 raft_consensus.cc:3032] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 15:54:40.905879 8136 raft_consensus.cc:729] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:40.906678 8136 consensus_queue.cc:260] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.920324 8136 raft_consensus.cc:3037] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 6 FOLLOWER]: Advancing to term 7
I20260807 15:54:40.924109 8134 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:40.924676 8134 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.928519 8134 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:40.930747 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.931401 7586 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 5.
I20260807 15:54:40.933513 7590 raft_consensus.cc:1226] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 LEADER]: Rejecting Update request from peer d77851634f654ce5bfce79a4dd797055 for earlier term 6. Current term is 7. Ops: []
I20260807 15:54:40.935992 7221 catalog_manager.cc:5495] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 6 to 7, leader changed from d77851634f654ce5bfce79a4dd797055 (127.5.45.3) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 7 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:40.942219 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:40.943341 8134 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 15:54:40.944599 8134 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:40.946044 8134 raft_consensus.cc:3037] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:40.947103 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:40.948249 7484 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 5: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:40.958207 8134 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.961195 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:40.962090 7484 raft_consensus.cc:2379] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 in current term 5: Already voted for candidate f07486fcec2343278bcc737786ff6fed in this term.
I20260807 15:54:40.966944 8134 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:40.967548 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:40.975615 8135 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:40.976125 8135 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.978164 8135 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:40.979841 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:40.980865 7685 raft_consensus.cc:2396] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260807 15:54:40.982107 7590 raft_consensus.cc:2454] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 5.
I20260807 15:54:40.983412 7637 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters: f07486fcec2343278bcc737786ff6fed
I20260807 15:54:40.984704 8134 raft_consensus.cc:2781] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 15:54:40.985173 8134 raft_consensus.cc:686] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 LEADER]: Becoming Leader. State: Replica: d77851634f654ce5bfce79a4dd797055, State: Running, Role: LEADER
I20260807 15:54:40.986088 8134 consensus_queue.cc:237] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:40.991273 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:40.992058 7586 raft_consensus.cc:2396] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260807 15:54:40.995968 7442 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:40.997160 8135 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:41.007701 7484 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260807 15:54:41.010111 8126 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:54:41.013238 7685 raft_consensus.cc:1261] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260807 15:54:41.013687 7685 pending_rounds.cc:76] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Aborting all ops after (but not including) 6
I20260807 15:54:41.013928 7685 pending_rounds.cc:97] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260807 15:54:41.014353 7685 raft_consensus.cc:2866] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 15:54:41.004006 7222 catalog_manager.cc:5495] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 reported cstate change: term changed from 4 to 5, leader changed from f847539a423f4000a7839152f3a1fa99 (127.5.45.2) to d77851634f654ce5bfce79a4dd797055 (127.5.45.3). New cstate: current_term: 5 leader_uuid: "d77851634f654ce5bfce79a4dd797055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: HEALTHY } } }
I20260807 15:54:41.018005 8144 consensus_queue.cc:1035] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:54:41.097429 7590 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 15:54:41.099543 8134 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:41.113376 7487 raft_consensus.cc:1261] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Refusing update from remote peer d77851634f654ce5bfce79a4dd797055: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 15:54:41.117648 8136 consensus_queue.cc:1035] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:41.252655 8174 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:41.253260 8174 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:41.255533 8174 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:41.257361 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:41.257850 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:41.259424 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:41.260186 8174 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 15:54:43.470340 8179 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:43.470950 8179 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:43.473570 8179 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:43.474687 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:43.475458 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:43.476853 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:43.477710 8179 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:43.831804 8179 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:43.832674 8179 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.003772 8185 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:44.012805 8186 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:44.005625 8179 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:44.012877 8185 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.014065 8186 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.009368 8182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:44.015805 8182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:44.016409 8185 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:44.022253 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:44.023150 8182 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:54:44.023846 8186 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
W20260807 15:54:44.067713 7441 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.45.3:34107 timed out after 0.232s (SENT) [suppressed 7 similar messages]
W20260807 15:54:44.317616 7441 leader_election.cc:334] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107): Timed out: RequestConsensusVote RPC to 127.5.45.3:34107 timed out after 0.232s (SENT)
W20260807 15:54:44.476243 7442 leader_election.cc:334] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.232s (SENT)
I20260807 15:54:44.476743 7442 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.478214 8186 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 15:54:44.502454 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800233d80 after lost signal to thread 5304
W20260807 15:54:44.503647 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800158de0 after lost signal to thread 7174
W20260807 15:54:44.504626 7515 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6580 after lost signal to thread 7267
W20260807 15:54:44.511422 7441 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107): Timed out: RequestConsensusVote RPC to 127.5.45.3:34107 timed out after 0.246s (SENT)
W20260807 15:54:44.513620 7442 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.246s (SENT)
I20260807 15:54:44.514127 7442 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.515230 8186 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 15:54:44.600835 7441 leader_election.cc:334] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107): Timed out: RequestConsensusVote RPC to 127.5.45.3:34107 timed out after 0.287s (SENT)
W20260807 15:54:44.601989 7442 leader_election.cc:334] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Timed out: RequestConsensusVote RPC to 127.5.45.2:37981 timed out after 0.287s (SENT)
I20260807 15:54:44.602494 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.603624 8186 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 15:54:44.617138 7111 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.45.60:42447, user_credentials={real_user=slave}} blocked reactor thread for 328769us
I20260807 15:54:44.690397 8186 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:44.691195 8186 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.694808 8186 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:44.724764 8186 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:44.725662 8186 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.729314 8186 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:44.812881 8189 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:44.814364 8189 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.817201 8189 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:44.820238 8190 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:44.821502 8190 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.826390 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:44.827109 7484 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 3.
I20260807 15:54:44.830829 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed; no voters:
I20260807 15:54:44.841251 8194 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 15:54:44.843007 8194 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:44.843658 8194 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 3 FOLLOWER]: Advancing to term 4
W20260807 15:54:44.882987 7111 leader_election.cc:334] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447): Timed out: RequestConsensusVote RPC to 127.5.45.60:42447 timed out after 0.264s (SENT)
I20260807 15:54:44.883670 7111 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
W20260807 15:54:44.894570 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080023dce0 after lost signal to thread 7518
I20260807 15:54:44.895852 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.904493 8190 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:44.905875 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
W20260807 15:54:44.910881 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800222380 after lost signal to thread 7618
W20260807 15:54:44.912685 7515 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5be0 after lost signal to thread 7712
W20260807 15:54:44.914348 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800267480 after lost signal to thread 8062
I20260807 15:54:44.917729 8196 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:44.918797 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.921192 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.923043 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.925239 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.926612 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
W20260807 15:54:44.926972 7199 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.344s user 0.010s sys 0.018s
W20260807 15:54:44.927480 7199 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.345s user 0.010s sys 0.018s
I20260807 15:54:44.928347 7686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.932582 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:44.928586 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:44.929916 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.930181 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:44.934381 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:54:44.932163 8191 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:54:44.936254 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.936977 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.936870 8194 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.939149 8194 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:44.942901 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:44.936254 8191 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:54:44.938153 7683 raft_consensus.cc:1226] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Rejecting Update request from peer f847539a423f4000a7839152f3a1fa99 for earlier term 3. Current term is 4. Ops: []
I20260807 15:54:44.947264 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.947340 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.948956 7589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.949239 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.950613 7542 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055, f07486fcec2343278bcc737786ff6fed
I20260807 15:54:44.951387 8125 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:44.940573 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:44.947765 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.947880 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:44.953753 7442 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.954561 8186 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:44.954808 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.955273 7442 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:44.955961 8186 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 15:54:44.955932 8194 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 15:54:44.961822 7147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "4dc450c6e3db439994750939886f3aa7" is_pre_election: true
I20260807 15:54:44.963286 8125 consensus_queue.cc:1046] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:44.964470 8189 raft_consensus.cc:3032] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 15:54:44.964831 8189 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:44.965158 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3acfcc965f048a4908d9687c07fd330" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:54:44.965634 8189 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:44.969440 7997 leader_election.cc:302] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3acfcc965f048a4908d9687c07fd330; no voters: 4dc450c6e3db439994750939886f3aa7, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:54:44.971393 8189 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:44.976816 8191 leader_election.cc:288] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661), df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549)
I20260807 15:54:44.977442 8191 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 15:54:45.136240 8186 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:45.136829 8186 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.138742 8186 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:45.139779 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:45.139998 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:45.140445 7591 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 4.
I20260807 15:54:45.141860 7683 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 4: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:45.142582 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:45.143560 8186 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 15:54:45.143990 8186 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Starting leader election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:45.144544 8186 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 3 FOLLOWER]: Advancing to term 4
I20260807 15:54:45.153090 8186 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.156375 8186 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 election: Requested vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:45.156570 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:45.158156 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:45.159024 7683 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f07486fcec2343278bcc737786ff6fed in current term 4: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:45.164492 7592 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07486fcec2343278bcc737786ff6fed in term 4.
I20260807 15:54:45.165822 7442 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:45.167007 8186 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Leader election won for term 4
I20260807 15:54:45.176290 8186 raft_consensus.cc:686] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 LEADER]: Becoming Leader. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:45.178385 8186 consensus_queue.cc:237] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.193038 7226 catalog_manager.cc:5495] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed reported cstate change: term changed from 3 to 4, leader changed from f847539a423f4000a7839152f3a1fa99 (127.5.45.2) to f07486fcec2343278bcc737786ff6fed (127.5.45.1). New cstate: current_term: 4 leader_uuid: "f07486fcec2343278bcc737786ff6fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:45.203307 8194 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:54:45.204042 8194 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.206308 8194 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:45.209061 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:45.210994 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:45.211935 7592 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77851634f654ce5bfce79a4dd797055 in term 4.
I20260807 15:54:45.215308 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:45.221714 8194 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 15:54:45.222299 8194 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:54:45.222895 8194 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:45.259889 7592 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Refusing update from remote peer f07486fcec2343278bcc737786ff6fed: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 15:54:45.261869 8186 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 15:54:45.266379 8194 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.273931 7683 raft_consensus.cc:1226] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Rejecting Update request from peer f07486fcec2343278bcc737786ff6fed for earlier term 4. Current term is 5. Ops: []
I20260807 15:54:45.276156 8186 consensus_queue.cc:1046] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 15:54:45.277434 8186 raft_consensus.cc:3032] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 LEADER]: Stepping down as leader of term 4
I20260807 15:54:45.277786 8186 raft_consensus.cc:729] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f07486fcec2343278bcc737786ff6fed, State: Running, Role: LEADER
I20260807 15:54:45.278652 8186 consensus_queue.cc:260] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.280193 8186 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:45.280736 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:45.288033 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99"
I20260807 15:54:45.286084 8194 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:45.299454 7487 raft_consensus.cc:2396] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d77851634f654ce5bfce79a4dd797055 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260807 15:54:45.309039 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:45.310431 8194 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 15:54:45.468147 8189 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:45.468676 8189 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.470512 8189 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:45.471318 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:45.471516 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:45.472029 7487 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 5.
I20260807 15:54:45.472275 7683 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 5: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:45.473189 7542 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters:
I20260807 15:54:45.473891 8189 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 15:54:45.474238 8189 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f07486fcec2343278bcc737786ff6fed)
I20260807 15:54:45.474584 8189 raft_consensus.cc:3037] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 4 FOLLOWER]: Advancing to term 5
I20260807 15:54:45.479687 8189 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.481266 8189 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 election: Requested vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:45.482069 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed"
I20260807 15:54:45.482199 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f847539a423f4000a7839152f3a1fa99" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055"
I20260807 15:54:45.483084 7683 raft_consensus.cc:2379] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f847539a423f4000a7839152f3a1fa99 in current term 5: Already voted for candidate d77851634f654ce5bfce79a4dd797055 in this term.
I20260807 15:54:45.487534 7487 raft_consensus.cc:2454] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f847539a423f4000a7839152f3a1fa99 in term 5.
I20260807 15:54:45.488466 7542 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99; no voters: d77851634f654ce5bfce79a4dd797055
I20260807 15:54:45.489079 8189 raft_consensus.cc:2781] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 15:54:45.489722 8189 raft_consensus.cc:686] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 5 LEADER]: Becoming Leader. State: Replica: f847539a423f4000a7839152f3a1fa99, State: Running, Role: LEADER
I20260807 15:54:45.490509 8189 consensus_queue.cc:237] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:45.497624 7226 catalog_manager.cc:5495] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 reported cstate change: term changed from 4 to 5, leader changed from f07486fcec2343278bcc737786ff6fed (127.5.45.1) to f847539a423f4000a7839152f3a1fa99 (127.5.45.2). New cstate: current_term: 5 leader_uuid: "f847539a423f4000a7839152f3a1fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } health_report { overall_health: UNKNOWN } } }
I20260807 15:54:45.578481 7683 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 15:54:45.579644 8189 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:45.592798 7487 raft_consensus.cc:1261] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Refusing update from remote peer f847539a423f4000a7839152f3a1fa99: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 15:54:45.595602 8217 consensus_queue.cc:1035] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:54:58.274370 8251 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:58.275439 8251 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:58.277884 8251 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:58.279151 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:58.289376 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:58.291505 7441 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:58.293745 8251 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:58.324239 8251 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:54:58.325122 8251 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:58.331445 8251 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:54:58.435379 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:54:58.436571 8231 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:58.437026 7592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:58.437153 8231 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:58.439705 8231 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:58.440752 7441 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:58.442484 8251 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:54:58.452661 8231 raft_consensus.cc:482] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:54:58.453408 8231 raft_consensus.cc:504] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:54:58.458846 8231 leader_election.cc:288] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:54:58.462162 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:58.464597 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:54:58.469511 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d3feeebcad478aa30ccdde78044663" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:58.471452 7484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2b29f18f024df199716835932639a1" candidate_uuid: "d77851634f654ce5bfce79a4dd797055" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f07486fcec2343278bcc737786ff6fed" is_pre_election: true
I20260807 15:54:58.471735 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:58.472630 8231 raft_consensus.cc:2726] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 15:54:58.473505 7637 leader_election.cc:302] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:54:58.474294 8231 raft_consensus.cc:2726] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 15:54:59.328357 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800267a20 after lost signal to thread 5304
W20260807 15:54:59.329511 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080022dbe0 after lost signal to thread 7174
W20260807 15:54:59.330268 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800126900 after lost signal to thread 7267
W20260807 15:54:59.331328 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800044cc0 after lost signal to thread 7518
W20260807 15:54:59.332211 7515 debug-util.cc:398] Leaking SignalData structure 0x7b08000a68c0 after lost signal to thread 7618
W20260807 15:54:59.333137 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800238180 after lost signal to thread 7712
W20260807 15:54:59.334143 7515 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad780 after lost signal to thread 8062
I20260807 15:55:01.982460 8261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader df6a37abab2d49529a2a2ca42d10c7cf)
I20260807 15:55:01.984045 8261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } }
I20260807 15:55:01.988510 8261 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers df6a37abab2d49529a2a2ca42d10c7cf (127.5.45.61:36549), a3acfcc965f048a4908d9687c07fd330 (127.5.45.60:42447)
I20260807 15:55:01.988911 7239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 25 } } ignore_live_leader: false dest_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" is_pre_election: true
I20260807 15:55:02.023136 8263 raft_consensus.cc:482] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d77851634f654ce5bfce79a4dd797055)
I20260807 15:55:02.023831 8263 raft_consensus.cc:504] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:55:02.028129 8263 leader_election.cc:288] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981), d77851634f654ce5bfce79a4dd797055 (127.5.45.3:34107)
I20260807 15:55:02.071121 7586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f847539a423f4000a7839152f3a1fa99" is_pre_election: true
I20260807 15:55:02.075205 7683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb581517aa04c7dbeba2ba716118eed" candidate_uuid: "f07486fcec2343278bcc737786ff6fed" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d77851634f654ce5bfce79a4dd797055" is_pre_election: true
I20260807 15:55:02.078009 7441 leader_election.cc:302] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07486fcec2343278bcc737786ff6fed; no voters: d77851634f654ce5bfce79a4dd797055, f847539a423f4000a7839152f3a1fa99
I20260807 15:55:02.080246 8263 raft_consensus.cc:2726] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 15:55:02.083254 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dc450c6e3db439994750939886f3aa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 25 } } ignore_live_leader: false dest_uuid: "a3acfcc965f048a4908d9687c07fd330" is_pre_election: true
I20260807 15:55:02.085386 7111 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dc450c6e3db439994750939886f3aa7; no voters: a3acfcc965f048a4908d9687c07fd330, df6a37abab2d49529a2a2ca42d10c7cf
I20260807 15:55:02.086992 8261 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 15:55:02.962687 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080022a520 after lost signal to thread 5304
W20260807 15:55:02.963763 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800110900 after lost signal to thread 7174
W20260807 15:55:02.964538 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800269700 after lost signal to thread 7267
W20260807 15:55:02.965727 7515 debug-util.cc:398] Leaking SignalData structure 0x7b0800234fe0 after lost signal to thread 7518
W20260807 15:55:02.966774 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080015efa0 after lost signal to thread 7618
W20260807 15:55:02.967618 7515 debug-util.cc:398] Leaking SignalData structure 0x7b080008c540 after lost signal to thread 7712
W20260807 15:55:02.968555 7515 debug-util.cc:398] Leaking SignalData structure 0x7b08000bda60 after lost signal to thread 8062
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:783: Failure
Expected equality of these values:
live_row_count
Which is: 135
table_info->GetMetrics()->live_row_count->value()
Which is: 0
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_util.cc:395: Failure
Failed
Timed out waiting for assertion to pass.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:767: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:785: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 782, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
Actual: it does.
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:895: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260807 15:55:04.442492 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
W20260807 15:55:04.445029 7542 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260807 15:55:04.469555 7637 consensus_peers.cc:487] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 -> Peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Couldn't send request to peer f07486fcec2343278bcc737786ff6fed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260807 15:55:04.473088 7542 consensus_peers.cc:487] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 -> Peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Couldn't send request to peer f07486fcec2343278bcc737786ff6fed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260807 15:55:04.492313 7542 consensus_peers.cc:487] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 -> Peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Couldn't send request to peer f07486fcec2343278bcc737786ff6fed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:04.504204 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:04.504925 5300 tablet_replica.cc:330] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed: stopping tablet replica
I20260807 15:55:04.505695 5300 raft_consensus.cc:2229] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.506343 5300 raft_consensus.cc:2258] T 7d2b29f18f024df199716835932639a1 P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.508855 5300 tablet_replica.cc:330] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed: stopping tablet replica
I20260807 15:55:04.509415 5300 raft_consensus.cc:2229] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.509938 5300 raft_consensus.cc:2258] T fcb581517aa04c7dbeba2ba716118eed P f07486fcec2343278bcc737786ff6fed [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.511958 5300 tablet_replica.cc:330] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed: stopping tablet replica
I20260807 15:55:04.512408 5300 raft_consensus.cc:2229] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.512787 5300 raft_consensus.cc:2258] T 21d3feeebcad478aa30ccdde78044663 P f07486fcec2343278bcc737786ff6fed [term 7 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.596516 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:04.617511 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:55:04.665194 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:04.666059 5300 tablet_replica.cc:330] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99: stopping tablet replica
I20260807 15:55:04.666709 5300 raft_consensus.cc:2229] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.667409 5300 raft_consensus.cc:2258] T fcb581517aa04c7dbeba2ba716118eed P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.669778 5300 tablet_replica.cc:330] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99: stopping tablet replica
I20260807 15:55:04.670286 5300 raft_consensus.cc:2229] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 LEADER]: Raft consensus shutting down.
I20260807 15:55:04.671195 5300 raft_consensus.cc:2258] T 21d3feeebcad478aa30ccdde78044663 P f847539a423f4000a7839152f3a1fa99 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.673463 5300 tablet_replica.cc:330] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99: stopping tablet replica
I20260807 15:55:04.673964 5300 raft_consensus.cc:2229] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 5 LEADER]: Raft consensus shutting down.
I20260807 15:55:04.674782 5300 raft_consensus.cc:2258] T 7d2b29f18f024df199716835932639a1 P f847539a423f4000a7839152f3a1fa99 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260807 15:55:04.684326 7637 consensus_peers.cc:487] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 -> Peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Couldn't send request to peer f847539a423f4000a7839152f3a1fa99. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:04.730595 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:55:04.750010 5300 tablet_server.cc:178] TabletServer@127.5.45.3:0 shutting down...
I20260807 15:55:04.790686 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:04.791369 5300 tablet_replica.cc:330] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055: stopping tablet replica
I20260807 15:55:04.791994 5300 raft_consensus.cc:2229] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.792574 5300 raft_consensus.cc:2258] T 7d2b29f18f024df199716835932639a1 P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.794327 8231 raft_consensus.cc:482] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f847539a423f4000a7839152f3a1fa99)
I20260807 15:55:04.794919 5300 tablet_replica.cc:330] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: stopping tablet replica
I20260807 15:55:04.794791 8231 raft_consensus.cc:504] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07486fcec2343278bcc737786ff6fed" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 42027 } } peers { permanent_uuid: "f847539a423f4000a7839152f3a1fa99" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 37981 } } peers { permanent_uuid: "d77851634f654ce5bfce79a4dd797055" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34107 } }
I20260807 15:55:04.795544 5300 raft_consensus.cc:2229] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.796052 5300 raft_consensus.cc:2258] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.796634 8231 leader_election.cc:288] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027), f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981)
I20260807 15:55:04.799750 5300 tablet_replica.cc:330] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055: stopping tablet replica
W20260807 15:55:04.800215 7637 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f07486fcec2343278bcc737786ff6fed (127.5.45.1:42027): Network error: Client connection negotiation failed: client connection to 127.5.45.1:42027: connect: Connection refused (error 111)
I20260807 15:55:04.800367 5300 raft_consensus.cc:2229] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 LEADER]: Raft consensus shutting down.
W20260807 15:55:04.800907 7637 leader_election.cc:334] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f847539a423f4000a7839152f3a1fa99 (127.5.45.2:37981): Network error: Client connection negotiation failed: client connection to 127.5.45.2:37981: connect: Connection refused (error 111)
I20260807 15:55:04.801257 7637 leader_election.cc:302] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d77851634f654ce5bfce79a4dd797055; no voters: f07486fcec2343278bcc737786ff6fed, f847539a423f4000a7839152f3a1fa99
I20260807 15:55:04.801448 5300 raft_consensus.cc:2258] T fcb581517aa04c7dbeba2ba716118eed P d77851634f654ce5bfce79a4dd797055 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260807 15:55:04.801884 7637 raft_consensus.cc:2670] T 21d3feeebcad478aa30ccdde78044663 P d77851634f654ce5bfce79a4dd797055: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260807 15:55:04.826052 5300 tablet_server.cc:194] TabletServer@127.5.45.3:0 shutdown complete.
I20260807 15:55:04.842857 5300 master.cc:537] Master@127.5.45.62:46661 shutting down...
W20260807 15:55:04.862834 7199 consensus_peers.cc:487] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf -> Peer 4dc450c6e3db439994750939886f3aa7 (127.5.45.62:46661): Couldn't send request to peer 4dc450c6e3db439994750939886f3aa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:04.875998 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:04.876654 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.877003 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4dc450c6e3db439994750939886f3aa7: stopping tablet replica
I20260807 15:55:04.896873 5300 master.cc:552] Master@127.5.45.62:46661 shutdown complete.
I20260807 15:55:04.918987 5300 master.cc:537] Master@127.5.45.61:36549 shutting down...
I20260807 15:55:04.936892 8252 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc450c6e3db439994750939886f3aa7" member_type: VOTER last_known_addr { host: "127.5.45.62" port: 46661 } } peers { permanent_uuid: "df6a37abab2d49529a2a2ca42d10c7cf" member_type: VOTER last_known_addr { host: "127.5.45.61" port: 36549 } } peers { permanent_uuid: "a3acfcc965f048a4908d9687c07fd330" member_type: VOTER last_known_addr { host: "127.5.45.60" port: 42447 } } }
I20260807 15:55:04.938117 8252 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:04.947486 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 3 LEADER]: Raft consensus shutting down.
I20260807 15:55:04.948386 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:04.948827 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P df6a37abab2d49529a2a2ca42d10c7cf: stopping tablet replica
I20260807 15:55:04.970839 5300 master.cc:552] Master@127.5.45.61:36549 shutdown complete.
I20260807 15:55:04.989498 5300 master.cc:537] Master@127.5.45.60:42447 shutting down...
I20260807 15:55:05.011942 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:05.012631 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a3acfcc965f048a4908d9687c07fd330 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:05.012976 5300 tablet_replica.cc:330] stopping tablet replica
I20260807 15:55:05.031935 5300 master.cc:552] Master@127.5.45.60:42447 shutdown complete.
I20260807 15:55:05.059368 5300 test_util.cc:175] -----------------------------------------------
I20260807 15:55:05.059614 5300 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786118049188364-5300-0
[ FAILED ] TsTabletManagerITest.TestTableStats (39107 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260807 15:55:05.066722 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:33309
I20260807 15:55:05.067878 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:05.073638 8290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.074788 8291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.076994 8294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:05.077448 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:05.078437 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:05.078625 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:05.078830 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118105078799 us; error 0 us; skew 500 ppm
I20260807 15:55:05.079445 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:05.085193 5300 webserver.cc:460] Webserver started at http://127.5.45.62:46741/ using document root <none> and password file <none>
I20260807 15:55:05.085855 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:05.086051 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:05.086428 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:05.087689 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "b56f79b886714b188f187b416a44f720"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.094218 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 15:55:05.098764 8300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.099721 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 15:55:05.100075 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "b56f79b886714b188f187b416a44f720"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.100374 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:05.122403 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:05.123694 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:05.165633 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:33309
I20260807 15:55:05.165743 8363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:33309 every 8 connection(s)
I20260807 15:55:05.169632 8365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:05.180833 8365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: Bootstrap starting.
I20260807 15:55:05.185405 8365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:05.190068 8365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: No bootstrap required, opened a new log
I20260807 15:55:05.192153 8365 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER }
I20260807 15:55:05.192538 8365 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:05.192719 8365 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56f79b886714b188f187b416a44f720, State: Initialized, Role: FOLLOWER
I20260807 15:55:05.193322 8365 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER }
I20260807 15:55:05.193773 8365 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:05.193959 8365 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:05.194191 8365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:05.198993 8365 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER }
I20260807 15:55:05.199566 8365 leader_election.cc:302] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56f79b886714b188f187b416a44f720; no voters:
I20260807 15:55:05.200855 8365 leader_election.cc:288] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:05.201419 8368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:05.203044 8368 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 1 LEADER]: Becoming Leader. State: Replica: b56f79b886714b188f187b416a44f720, State: Running, Role: LEADER
I20260807 15:55:05.203768 8368 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER }
I20260807 15:55:05.204092 8365 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:05.207693 8369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56f79b886714b188f187b416a44f720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER } }
I20260807 15:55:05.208316 8369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:05.209409 8370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56f79b886714b188f187b416a44f720. Latest consensus state: current_term: 1 leader_uuid: "b56f79b886714b188f187b416a44f720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56f79b886714b188f187b416a44f720" member_type: VOTER } }
I20260807 15:55:05.210203 8370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:05.215883 8379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:05.220814 8379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:05.225039 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:05.230149 8379 catalog_manager.cc:1347] Generated new cluster ID: c9010ad4284744efabe17b9e3ecbd4c6
I20260807 15:55:05.230436 8379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:05.253108 8379 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:05.254853 8379 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:05.273300 8379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: Generated new TSK 0
I20260807 15:55:05.274049 8379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:05.292593 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:05.299247 8390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.300267 8393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.301887 8396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:05.302489 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:05.303682 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:05.303859 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:05.304047 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118105304023 us; error 0 us; skew 500 ppm
I20260807 15:55:05.304587 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:05.307139 5300 webserver.cc:460] Webserver started at http://127.5.45.1:35675/ using document root <none> and password file <none>
I20260807 15:55:05.307646 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:05.307874 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:05.308157 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:05.309458 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "4e3bef8306814fffa25b1dd5bdb81bfb"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.314041 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:55:05.317332 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.318281 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:55:05.318598 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "4e3bef8306814fffa25b1dd5bdb81bfb"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.318936 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:05.337487 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:05.338706 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:05.340260 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:05.342650 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:05.342871 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.343117 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:05.343307 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.384737 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:44617
I20260807 15:55:05.384864 8483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:44617 every 8 connection(s)
I20260807 15:55:05.389801 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:05.397653 8490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.400900 8491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.401419 8493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:05.402271 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:05.403743 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:05.404091 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:05.404338 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118105404323 us; error 0 us; skew 500 ppm
I20260807 15:55:05.405089 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:05.406100 8484 heartbeater.cc:347] Connected to a master server at 127.5.45.62:33309
I20260807 15:55:05.406553 8484 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:05.407320 8484 heartbeater.cc:511] Master 127.5.45.62:33309 requested a full tablet report, sending...
I20260807 15:55:05.407888 5300 webserver.cc:460] Webserver started at http://127.5.45.2:44979/ using document root <none> and password file <none>
I20260807 15:55:05.408411 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:05.408625 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:05.408941 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:05.409595 8321 ts_manager.cc:194] Registered new tserver with Master: 4e3bef8306814fffa25b1dd5bdb81bfb (127.5.45.1:44617)
I20260807 15:55:05.410516 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.412415 8321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49792
I20260807 15:55:05.416023 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 15:55:05.419530 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.420331 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:55:05.420691 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.421056 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:05.461581 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:05.462832 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:05.464321 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:05.466588 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:05.466809 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.467057 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:05.467236 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.508389 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:34789
I20260807 15:55:05.508527 8579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:34789 every 8 connection(s)
I20260807 15:55:05.513433 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:05.521056 8583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.523411 8584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:05.525418 8588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:05.526742 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:05.527724 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:05.528002 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:05.528220 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118105528202 us; error 0 us; skew 500 ppm
I20260807 15:55:05.528594 8580 heartbeater.cc:347] Connected to a master server at 127.5.45.62:33309
I20260807 15:55:05.528916 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:05.529012 8580 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:05.530051 8580 heartbeater.cc:511] Master 127.5.45.62:33309 requested a full tablet report, sending...
I20260807 15:55:05.532002 5300 webserver.cc:460] Webserver started at http://127.5.45.3:35609/ using document root <none> and password file <none>
I20260807 15:55:05.532373 8321 ts_manager.cc:194] Registered new tserver with Master: 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789)
I20260807 15:55:05.532686 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:05.532960 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:05.533269 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:05.534668 8321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49798
I20260807 15:55:05.534780 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root/instance:
uuid: "53c21020fb4c47759d6587598d3d329b"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.539781 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:55:05.543352 8593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.544282 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:55:05.544605 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root
uuid: "53c21020fb4c47759d6587598d3d329b"
format_stamp: "Formatted at 2026-08-07 15:55:05 on dist-test-slave-bw88"
I20260807 15:55:05.544925 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:05.580137 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:05.581522 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:05.583082 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:05.585397 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:05.585619 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.585875 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:05.586042 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:05.630950 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.3:40197
I20260807 15:55:05.631094 8671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.3:40197 every 8 connection(s)
I20260807 15:55:05.646739 8672 heartbeater.cc:347] Connected to a master server at 127.5.45.62:33309
I20260807 15:55:05.647132 8672 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:05.647855 8672 heartbeater.cc:511] Master 127.5.45.62:33309 requested a full tablet report, sending...
I20260807 15:55:05.649778 8321 ts_manager.cc:194] Registered new tserver with Master: 53c21020fb4c47759d6587598d3d329b (127.5.45.3:40197)
I20260807 15:55:05.650200 5300 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016013423s
I20260807 15:55:05.651475 8321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49804
I20260807 15:55:05.688263 8321 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260807 15:55:05.690065 8321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:55:05.733990 8443 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 (DEFAULT_TABLE table=test-table [id=30374fdf27004caebfcfea38c5962b0f]), partition=
I20260807 15:55:05.735185 8443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6afa6a421749bd9f02ff2acf95d584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:05.744027 8536 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 (DEFAULT_TABLE table=test-table [id=30374fdf27004caebfcfea38c5962b0f]), partition=
I20260807 15:55:05.745230 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6afa6a421749bd9f02ff2acf95d584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:05.746662 8630 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 (DEFAULT_TABLE table=test-table [id=30374fdf27004caebfcfea38c5962b0f]), partition=
I20260807 15:55:05.747797 8630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6afa6a421749bd9f02ff2acf95d584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:05.750437 8692 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Bootstrap starting.
I20260807 15:55:05.755401 8692 tablet_bootstrap.cc:654] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:05.760464 8692 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: No bootstrap required, opened a new log
I20260807 15:55:05.760921 8692 ts_tablet_manager.cc:1397] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 15:55:05.762146 8694 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Bootstrap starting.
I20260807 15:55:05.763991 8692 raft_consensus.cc:348] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.764904 8692 raft_consensus.cc:374] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:05.765286 8692 raft_consensus.cc:729] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3bef8306814fffa25b1dd5bdb81bfb, State: Initialized, Role: FOLLOWER
I20260807 15:55:05.766021 8692 consensus_queue.cc:260] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.768745 8694 tablet_bootstrap.cc:654] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:05.768965 8484 heartbeater.cc:503] Master 127.5.45.62:33309 was elected leader, sending a full tablet report...
I20260807 15:55:05.769217 8692 ts_tablet_manager.cc:1428] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260807 15:55:05.769773 8692 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 15:55:05.771378 8695 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Bootstrap starting.
I20260807 15:55:05.774914 8694 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: No bootstrap required, opened a new log
I20260807 15:55:05.775410 8694 ts_tablet_manager.cc:1397] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260807 15:55:05.777865 8695 tablet_bootstrap.cc:654] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:05.778357 8694 raft_consensus.cc:348] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.779157 8694 raft_consensus.cc:374] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:05.779445 8694 raft_consensus.cc:729] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1dbc5e7ad94da2b823e79adcfa12a0, State: Initialized, Role: FOLLOWER
I20260807 15:55:05.780190 8694 consensus_queue.cc:260] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.782569 8580 heartbeater.cc:503] Master 127.5.45.62:33309 was elected leader, sending a full tablet report...
I20260807 15:55:05.783123 8695 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: No bootstrap required, opened a new log
I20260807 15:55:05.783100 8694 ts_tablet_manager.cc:1428] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
W20260807 15:55:05.783697 8694 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 15:55:05.783818 8695 ts_tablet_manager.cc:1397] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260807 15:55:05.785965 8695 raft_consensus.cc:348] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.786537 8695 raft_consensus.cc:374] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:05.786808 8695 raft_consensus.cc:729] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c21020fb4c47759d6587598d3d329b, State: Initialized, Role: FOLLOWER
I20260807 15:55:05.787533 8695 consensus_queue.cc:260] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.789707 8672 heartbeater.cc:503] Master 127.5.45.62:33309 was elected leader, sending a full tablet report...
I20260807 15:55:05.790107 8695 ts_tablet_manager.cc:1428] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 15:55:05.790392 8695 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
W20260807 15:55:05.884874 8673 tablet.cc:2369] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:55:05.900803 8696 raft_consensus.cc:482] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:55:05.901434 8696 raft_consensus.cc:504] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.903365 8696 leader_election.cc:288] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789), 53c21020fb4c47759d6587598d3d329b (127.5.45.3:40197)
I20260807 15:55:05.916653 8548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6afa6a421749bd9f02ff2acf95d584" candidate_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" is_pre_election: true
I20260807 15:55:05.917527 8548 raft_consensus.cc:2454] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e3bef8306814fffa25b1dd5bdb81bfb in term 0.
I20260807 15:55:05.918217 8643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6afa6a421749bd9f02ff2acf95d584" candidate_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c21020fb4c47759d6587598d3d329b" is_pre_election: true
I20260807 15:55:05.918897 8643 raft_consensus.cc:2454] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e3bef8306814fffa25b1dd5bdb81bfb in term 0.
I20260807 15:55:05.918809 8407 leader_election.cc:302] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3bef8306814fffa25b1dd5bdb81bfb, 8b1dbc5e7ad94da2b823e79adcfa12a0; no voters:
I20260807 15:55:05.919875 8696 raft_consensus.cc:2781] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:55:05.920243 8696 raft_consensus.cc:482] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:55:05.920534 8696 raft_consensus.cc:3037] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:05.925433 8696 raft_consensus.cc:504] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.927101 8696 leader_election.cc:288] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [CANDIDATE]: Term 1 election: Requested vote from peers 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789), 53c21020fb4c47759d6587598d3d329b (127.5.45.3:40197)
I20260807 15:55:05.927883 8548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6afa6a421749bd9f02ff2acf95d584" candidate_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0"
I20260807 15:55:05.928130 8643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6afa6a421749bd9f02ff2acf95d584" candidate_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c21020fb4c47759d6587598d3d329b"
I20260807 15:55:05.928390 8548 raft_consensus.cc:3037] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:05.928720 8643 raft_consensus.cc:3037] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:05.933882 8548 raft_consensus.cc:2454] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e3bef8306814fffa25b1dd5bdb81bfb in term 1.
I20260807 15:55:05.934062 8643 raft_consensus.cc:2454] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e3bef8306814fffa25b1dd5bdb81bfb in term 1.
I20260807 15:55:05.934976 8407 leader_election.cc:302] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3bef8306814fffa25b1dd5bdb81bfb, 8b1dbc5e7ad94da2b823e79adcfa12a0; no voters:
I20260807 15:55:05.935824 8696 raft_consensus.cc:2781] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:05.936908 8696 raft_consensus.cc:686] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 LEADER]: Becoming Leader. State: Replica: 4e3bef8306814fffa25b1dd5bdb81bfb, State: Running, Role: LEADER
I20260807 15:55:05.937719 8696 consensus_queue.cc:237] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:05.945113 8321 catalog_manager.cc:5495] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e3bef8306814fffa25b1dd5bdb81bfb (127.5.45.1). New cstate: current_term: 1 leader_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } health_report { overall_health: UNKNOWN } } }
I20260807 15:55:06.017813 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:60738
I20260807 15:55:06.020211 8706 tablet_replica.cc:330] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: stopping tablet replica
I20260807 15:55:06.021173 8706 raft_consensus.cc:2229] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:06.021831 8706 raft_consensus.cc:2258] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.024863 8706 ts_tablet_manager.cc:1905] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 15:55:06.043488 8706 ts_tablet_manager.cc:1918] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260807 15:55:06.044032 8706 log.cc:1196] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root/wals/4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.056210 8708 ts_tablet_manager.cc:927] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Initiating tablet copy from peer 4e3bef8306814fffa25b1dd5bdb81bfb (127.5.45.1:44617)
I20260807 15:55:06.058334 8708 tablet_copy_client.cc:316] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet copy: Beginning tablet copy session from remote peer at address 127.5.45.1:44617
I20260807 15:55:06.072858 8467 tablet_copy_service.cc:140] P 4e3bef8306814fffa25b1dd5bdb81bfb: Received BeginTabletCopySession request for tablet 4a6afa6a421749bd9f02ff2acf95d584 from peer 8b1dbc5e7ad94da2b823e79adcfa12a0 ({username='slave'} at 127.0.0.1:54710)
I20260807 15:55:06.073534 8467 tablet_copy_service.cc:161] P 4e3bef8306814fffa25b1dd5bdb81bfb: Beginning new tablet copy session on tablet 4a6afa6a421749bd9f02ff2acf95d584 from peer 8b1dbc5e7ad94da2b823e79adcfa12a0 at {username='slave'} at 127.0.0.1:54710: session id = 8b1dbc5e7ad94da2b823e79adcfa12a0-4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.079806 8467 tablet_copy_source_session.cc:205] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:55:06.082711 8708 ts_tablet_manager.cc:1905] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260807 15:55:06.093115 8708 ts_tablet_manager.cc:1918] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260807 15:55:06.093845 8708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6afa6a421749bd9f02ff2acf95d584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:06.099336 8708 tablet_copy_client.cc:791] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet copy: Starting download of 0 data blocks...
I20260807 15:55:06.099915 8708 tablet_copy_client.cc:655] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet copy: Starting download of 1 WAL segments...
I20260807 15:55:06.102048 8321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49824:
table { table_id: "30374fdf27004caebfcfea38c5962b0f" table_name: "test-table" }
I20260807 15:55:06.102897 8321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49824:
table { table_id: "30374fdf27004caebfcfea38c5962b0f" table_name: "test-table" }
I20260807 15:55:06.104542 8708 tablet_copy_client.cc:531] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:55:06.111369 8708 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Bootstrap starting.
I20260807 15:55:06.116742 8321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: Sending DeleteTablet for 3 replicas of tablet 4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.119248 8443 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:55:06 UTC) from {username='slave'} at 127.0.0.1:54694
I20260807 15:55:06.119467 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:55:06 UTC) from {username='slave'} at 127.0.0.1:60726
I20260807 15:55:06.121616 8712 tablet_replica.cc:330] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: stopping tablet replica
I20260807 15:55:06.121881 8630 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:55:06 UTC) from {username='slave'} at 127.0.0.1:37724
I20260807 15:55:06.122462 8712 raft_consensus.cc:2229] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 LEADER]: Raft consensus shutting down.
W20260807 15:55:06.122735 8303 catalog_manager.cc:4821] TS 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789): delete failed for tablet 4a6afa6a421749bd9f02ff2acf95d584 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a6afa6a421749bd9f02ff2acf95d584 already in progress: copying tablet
I20260807 15:55:06.123385 8712 pending_rounds.cc:62] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Trying to abort 1 pending ops.
I20260807 15:55:06.123698 8712 pending_rounds.cc:68] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315939761928060928 op_type: NO_OP noop_request { }
I20260807 15:55:06.124222 8712 raft_consensus.cc:2866] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 15:55:06.124614 8712 raft_consensus.cc:2258] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.128866 8712 ts_tablet_manager.cc:1905] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:55:06.134229 8713 tablet_replica.cc:330] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: stopping tablet replica
I20260807 15:55:06.135551 8713 raft_consensus.cc:2229] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:06.135901 8708 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 15:55:06.136171 8713 raft_consensus.cc:2258] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.136993 8708 tablet_bootstrap.cc:492] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Bootstrap complete.
I20260807 15:55:06.137812 8708 ts_tablet_manager.cc:1397] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.015s
I20260807 15:55:06.139297 8713 ts_tablet_manager.cc:1905] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:55:06.140645 8708 raft_consensus.cc:348] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:06.141727 8708 raft_consensus.cc:729] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1dbc5e7ad94da2b823e79adcfa12a0, State: Initialized, Role: FOLLOWER
I20260807 15:55:06.142616 8708 consensus_queue.cc:260] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3bef8306814fffa25b1dd5bdb81bfb" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 44617 } } peers { permanent_uuid: "8b1dbc5e7ad94da2b823e79adcfa12a0" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 34789 } } peers { permanent_uuid: "53c21020fb4c47759d6587598d3d329b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 40197 } }
I20260807 15:55:06.146624 8708 ts_tablet_manager.cc:1428] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260807 15:55:06.146911 8708 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260807 15:55:06.149593 8712 ts_tablet_manager.cc:1918] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 15:55:06.150098 8712 log.cc:1196] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-0-root/wals/4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.150264 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:55:06 UTC) from {username='slave'} at 127.0.0.1:60726
I20260807 15:55:06.150955 8712 ts_tablet_manager.cc:1939] T 4a6afa6a421749bd9f02ff2acf95d584 P 4e3bef8306814fffa25b1dd5bdb81bfb: Deleting consensus metadata
W20260807 15:55:06.152024 8303 catalog_manager.cc:4821] TS 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789): delete failed for tablet 4a6afa6a421749bd9f02ff2acf95d584 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a6afa6a421749bd9f02ff2acf95d584 already in progress: copying tablet
I20260807 15:55:06.152699 8302 catalog_manager.cc:4841] TS 4e3bef8306814fffa25b1dd5bdb81bfb (127.5.45.1:44617): tablet 4a6afa6a421749bd9f02ff2acf95d584 (table test-table [id=30374fdf27004caebfcfea38c5962b0f]) successfully deleted
I20260807 15:55:06.160946 8713 ts_tablet_manager.cc:1918] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 15:55:06.161510 8713 log.cc:1196] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-2-root/wals/4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.162338 8713 ts_tablet_manager.cc:1939] T 4a6afa6a421749bd9f02ff2acf95d584 P 53c21020fb4c47759d6587598d3d329b: Deleting consensus metadata
I20260807 15:55:06.164988 8301 catalog_manager.cc:4841] TS 53c21020fb4c47759d6587598d3d329b (127.5.45.3:40197): tablet 4a6afa6a421749bd9f02ff2acf95d584 (table test-table [id=30374fdf27004caebfcfea38c5962b0f]) successfully deleted
I20260807 15:55:06.199399 8467 tablet_copy_service.cc:342] P 4e3bef8306814fffa25b1dd5bdb81bfb: Request end of tablet copy session 8b1dbc5e7ad94da2b823e79adcfa12a0-4a6afa6a421749bd9f02ff2acf95d584 received from {username='slave'} at 127.0.0.1:54710
I20260807 15:55:06.199877 8467 tablet_copy_service.cc:433] P 4e3bef8306814fffa25b1dd5bdb81bfb: ending tablet copy session 8b1dbc5e7ad94da2b823e79adcfa12a0-4a6afa6a421749bd9f02ff2acf95d584 on tablet 4a6afa6a421749bd9f02ff2acf95d584 with peer 8b1dbc5e7ad94da2b823e79adcfa12a0
I20260807 15:55:06.203579 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6afa6a421749bd9f02ff2acf95d584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:55:06 UTC) from {username='slave'} at 127.0.0.1:60726
I20260807 15:55:06.204481 8706 tablet_replica.cc:330] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: stopping tablet replica
I20260807 15:55:06.205147 8706 raft_consensus.cc:2229] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:06.205628 8706 pending_rounds.cc:62] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Trying to abort 1 pending ops.
I20260807 15:55:06.205893 8706 pending_rounds.cc:68] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315939761928060928 op_type: NO_OP noop_request { }
I20260807 15:55:06.206262 8706 raft_consensus.cc:2866] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 15:55:06.206615 8706 raft_consensus.cc:2258] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.208877 8706 ts_tablet_manager.cc:1905] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:55:06.219061 8706 ts_tablet_manager.cc:1918] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 15:55:06.219415 8706 log.cc:1196] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786118049188364-5300-0/minicluster-data/ts-1-root/wals/4a6afa6a421749bd9f02ff2acf95d584
I20260807 15:55:06.220034 8706 ts_tablet_manager.cc:1939] T 4a6afa6a421749bd9f02ff2acf95d584 P 8b1dbc5e7ad94da2b823e79adcfa12a0: Deleting consensus metadata
I20260807 15:55:06.222589 8303 catalog_manager.cc:4841] TS 8b1dbc5e7ad94da2b823e79adcfa12a0 (127.5.45.2:34789): tablet 4a6afa6a421749bd9f02ff2acf95d584 (table test-table [id=30374fdf27004caebfcfea38c5962b0f]) successfully deleted
I20260807 15:55:06.276983 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:06.298586 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:06.317616 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:06.327344 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:55:06.345633 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:06.363011 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:55:06.372339 5300 tablet_server.cc:178] TabletServer@127.5.45.3:0 shutting down...
I20260807 15:55:06.391610 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:06.408196 5300 tablet_server.cc:194] TabletServer@127.5.45.3:0 shutdown complete.
I20260807 15:55:06.416630 5300 master.cc:537] Master@127.5.45.62:33309 shutting down...
I20260807 15:55:06.433177 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:06.433808 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.434137 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P b56f79b886714b188f187b416a44f720: stopping tablet replica
I20260807 15:55:06.453517 5300 master.cc:552] Master@127.5.45.62:33309 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1411 ms)
[----------] 4 tests from TsTabletManagerITest (46180 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260807 15:55:06.478758 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:43809
I20260807 15:55:06.479849 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:06.484764 8720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:06.486300 8721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:06.487685 8726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:06.488253 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:06.488929 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:06.489099 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:06.489207 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118106489198 us; error 0 us; skew 500 ppm
I20260807 15:55:06.489782 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:06.492039 5300 webserver.cc:460] Webserver started at http://127.5.45.62:37965/ using document root <none> and password file <none>
I20260807 15:55:06.492467 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:06.492627 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:06.492830 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:06.493940 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "c08c5cf6b9734fba9b102aafbe87be0d"
format_stamp: "Formatted at 2026-08-07 15:55:06 on dist-test-slave-bw88"
I20260807 15:55:06.498283 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 15:55:06.501727 8733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:06.502583 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 15:55:06.502902 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "c08c5cf6b9734fba9b102aafbe87be0d"
format_stamp: "Formatted at 2026-08-07 15:55:06 on dist-test-slave-bw88"
I20260807 15:55:06.503196 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:06.521129 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:06.522251 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:06.558911 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:43809
I20260807 15:55:06.559018 8797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:43809 every 8 connection(s)
I20260807 15:55:06.562883 8798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:06.572991 8798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d: Bootstrap starting.
I20260807 15:55:06.577512 8798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:06.581614 8798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d: No bootstrap required, opened a new log
I20260807 15:55:06.583470 8798 raft_consensus.cc:348] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER }
I20260807 15:55:06.583856 8798 raft_consensus.cc:374] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:06.584030 8798 raft_consensus.cc:729] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08c5cf6b9734fba9b102aafbe87be0d, State: Initialized, Role: FOLLOWER
I20260807 15:55:06.584520 8798 consensus_queue.cc:260] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER }
I20260807 15:55:06.584928 8798 raft_consensus.cc:388] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:06.585108 8798 raft_consensus.cc:482] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:06.585318 8798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:06.589709 8798 raft_consensus.cc:504] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER }
I20260807 15:55:06.590173 8798 leader_election.cc:302] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c08c5cf6b9734fba9b102aafbe87be0d; no voters:
I20260807 15:55:06.591252 8798 leader_election.cc:288] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:06.591634 8801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:06.593056 8801 raft_consensus.cc:686] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 1 LEADER]: Becoming Leader. State: Replica: c08c5cf6b9734fba9b102aafbe87be0d, State: Running, Role: LEADER
I20260807 15:55:06.593763 8801 consensus_queue.cc:237] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER }
I20260807 15:55:06.594478 8798 sys_catalog.cc:564] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:06.596853 8803 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c08c5cf6b9734fba9b102aafbe87be0d. Latest consensus state: current_term: 1 leader_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER } }
I20260807 15:55:06.597424 8803 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:06.601687 8802 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08c5cf6b9734fba9b102aafbe87be0d" member_type: VOTER } }
I20260807 15:55:06.602746 8802 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:06.608016 8806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:06.612581 8806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:06.615049 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:06.620909 8806 catalog_manager.cc:1347] Generated new cluster ID: 5cf473a1dd774bf7ad907467a12e0a01
I20260807 15:55:06.621207 8806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:06.634176 8806 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:06.635926 8806 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:06.646175 8806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d: Generated new TSK 0
I20260807 15:55:06.646852 8806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:06.682228 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:06.688048 8822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:06.689558 8823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:06.690907 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:06.691284 8825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:06.692215 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:06.692440 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:06.692564 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118106692555 us; error 0 us; skew 500 ppm
I20260807 15:55:06.693019 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:06.695449 5300 webserver.cc:460] Webserver started at http://127.5.45.1:41147/ using document root <none> and password file <none>
I20260807 15:55:06.695977 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:06.696144 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:06.696349 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:06.697491 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "5afa5930cb0946749567f3e61f8e0400"
format_stamp: "Formatted at 2026-08-07 15:55:06 on dist-test-slave-bw88"
I20260807 15:55:06.701998 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:55:06.705482 8831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:06.706233 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:55:06.706487 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "5afa5930cb0946749567f3e61f8e0400"
format_stamp: "Formatted at 2026-08-07 15:55:06 on dist-test-slave-bw88"
I20260807 15:55:06.706744 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:06.721590 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:06.722669 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:06.729403 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:06.729748 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:06.730099 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:06.730352 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:06.783890 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46325
I20260807 15:55:06.783990 8914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46325 every 8 connection(s)
I20260807 15:55:06.788516 8915 heartbeater.cc:347] Connected to a master server at 127.5.45.62:43809
I20260807 15:55:06.788885 8915 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:06.789669 8915 heartbeater.cc:511] Master 127.5.45.62:43809 requested a full tablet report, sending...
I20260807 15:55:06.791539 8752 ts_manager.cc:194] Registered new tserver with Master: 5afa5930cb0946749567f3e61f8e0400 (127.5.45.1:46325)
I20260807 15:55:06.792068 5300 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00495989s
I20260807 15:55:06.794226 8752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39122
I20260807 15:55:06.828986 8874 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 15:55:06.830472 8874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:06.842705 8933 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Bootstrap starting.
I20260807 15:55:06.846982 8933 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:06.850979 8933 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: No bootstrap required, opened a new log
I20260807 15:55:06.851328 8933 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 15:55:06.853089 8933 raft_consensus.cc:348] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:06.853559 8933 raft_consensus.cc:374] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:06.853756 8933 raft_consensus.cc:729] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5afa5930cb0946749567f3e61f8e0400, State: Initialized, Role: FOLLOWER
I20260807 15:55:06.854171 8933 consensus_queue.cc:260] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:06.854573 8933 raft_consensus.cc:388] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:06.854761 8933 raft_consensus.cc:482] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:06.854966 8933 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:06.859447 8933 raft_consensus.cc:504] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:06.859941 8933 leader_election.cc:302] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5afa5930cb0946749567f3e61f8e0400; no voters:
I20260807 15:55:06.861002 8933 leader_election.cc:288] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:06.861447 8935 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:06.863052 8915 heartbeater.cc:503] Master 127.5.45.62:43809 was elected leader, sending a full tablet report...
I20260807 15:55:06.863418 8933 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 15:55:06.863387 8935 raft_consensus.cc:686] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 LEADER]: Becoming Leader. State: Replica: 5afa5930cb0946749567f3e61f8e0400, State: Running, Role: LEADER
I20260807 15:55:06.864253 8935 consensus_queue.cc:237] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:06.868477 8936 tablet_replica.cc:439] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5afa5930cb0946749567f3e61f8e0400" committed_config { opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } } }
I20260807 15:55:06.869094 8936 tablet_replica.cc:442] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:06.869004 8937 tablet_replica.cc:439] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: TxnStatusTablet state changed. Reason: New leader 5afa5930cb0946749567f3e61f8e0400. Latest consensus state: current_term: 1 leader_uuid: "5afa5930cb0946749567f3e61f8e0400" committed_config { opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } } }
I20260807 15:55:06.869613 8937 tablet_replica.cc:442] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: This TxnStatusTablet replica's current role is: LEADER
W20260807 15:55:06.872035 8752 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:06.875453 8939 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:06.875948 8939 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 15:55:06.876962 8752 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:06.908522 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:06.933010 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:06.933967 5300 tablet_replica.cc:330] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: stopping tablet replica
I20260807 15:55:06.934665 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:06.935281 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:06.949599 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:06.965636 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:06.972474 8944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:06.972844 8945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:06.976810 8947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:06.976925 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:06.978127 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:06.978303 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:06.978518 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118106978486 us; error 0 us; skew 500 ppm
I20260807 15:55:06.979079 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:06.982007 5300 webserver.cc:460] Webserver started at http://127.5.45.1:41147/ using document root <none> and password file <none>
I20260807 15:55:06.982504 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:06.982651 5300 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 15:55:06.986536 5300 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 15:55:06.989570 8952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:06.990495 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:55:06.990800 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "5afa5930cb0946749567f3e61f8e0400"
format_stamp: "Formatted at 2026-08-07 15:55:06 on dist-test-slave-bw88"
I20260807 15:55:06.991114 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:07.012346 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:07.013518 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:07.021442 8963 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 15:55:07.029822 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:55:07.030112 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260807 15:55:07.030431 5300 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:55:07.036010 5300 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:55:07.036252 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.006s user 0.004s sys 0.000s
I20260807 15:55:07.036543 8963 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Bootstrap starting.
I20260807 15:55:07.081174 8963 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:55:07.082562 8963 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Bootstrap complete.
I20260807 15:55:07.083389 8963 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Time spent bootstrapping tablet: real 0.047s user 0.036s sys 0.008s
I20260807 15:55:07.086843 8963 raft_consensus.cc:348] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:07.087556 8963 raft_consensus.cc:729] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5afa5930cb0946749567f3e61f8e0400, State: Initialized, Role: FOLLOWER
I20260807 15:55:07.088640 8963 consensus_queue.cc:260] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:07.089506 8963 raft_consensus.cc:388] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:07.089906 8963 raft_consensus.cc:482] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:07.090317 8963 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:55:07.100618 8963 raft_consensus.cc:504] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:07.101708 8963 leader_election.cc:302] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5afa5930cb0946749567f3e61f8e0400; no voters:
I20260807 15:55:07.117713 8963 leader_election.cc:288] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:55:07.117964 8987 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:55:07.125222 8963 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400: Time spent starting tablet: real 0.041s user 0.028s sys 0.011s
I20260807 15:55:07.126675 8987 raft_consensus.cc:686] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 2 LEADER]: Becoming Leader. State: Replica: 5afa5930cb0946749567f3e61f8e0400, State: Running, Role: LEADER
I20260807 15:55:07.127555 8987 consensus_queue.cc:237] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } }
I20260807 15:55:07.137411 8991 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 5afa5930cb0946749567f3e61f8e0400. Latest consensus state: current_term: 2 leader_uuid: "5afa5930cb0946749567f3e61f8e0400" committed_config { opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } } }
I20260807 15:55:07.137506 8990 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5afa5930cb0946749567f3e61f8e0400" committed_config { opid_index: -1 peers { permanent_uuid: "5afa5930cb0946749567f3e61f8e0400" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46325 } } }
I20260807 15:55:07.138221 8991 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:07.138249 8990 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:07.142836 9006 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:07.143473 9006 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:55:07.172356 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46325
I20260807 15:55:07.172627 9047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46325 every 8 connection(s)
I20260807 15:55:07.178808 9048 heartbeater.cc:347] Connected to a master server at 127.5.45.62:43809
I20260807 15:55:07.179430 9048 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:07.180181 5300 tablet_server.cc:178] TabletServer@127.5.45.1:46325 shutting down...
I20260807 15:55:07.180979 9048 heartbeater.cc:511] Master 127.5.45.62:43809 requested a full tablet report, sending...
I20260807 15:55:07.191710 8752 ts_manager.cc:194] Re-registered known tserver with Master: 5afa5930cb0946749567f3e61f8e0400 (127.5.45.1:46325)
W20260807 15:55:07.192420 8752 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:07.194545 8752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39146
I20260807 15:55:07.198812 9048 heartbeater.cc:503] Master 127.5.45.62:43809 was elected leader, sending a full tablet report...
W20260807 15:55:07.202868 8752 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:07.215626 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:07.216502 5300 tablet_replica.cc:330] stopping tablet replica
I20260807 15:55:07.217078 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 2 LEADER]: Raft consensus shutting down.
I20260807 15:55:07.217590 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 5afa5930cb0946749567f3e61f8e0400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:07.238922 5300 tablet_server.cc:194] TabletServer@127.5.45.1:46325 shutdown complete.
I20260807 15:55:07.251360 5300 master.cc:537] Master@127.5.45.62:43809 shutting down...
I20260807 15:55:07.272346 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:07.272984 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:07.273411 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P c08c5cf6b9734fba9b102aafbe87be0d: stopping tablet replica
I20260807 15:55:07.293032 5300 master.cc:552] Master@127.5.45.62:43809 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (839 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260807 15:55:07.319227 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:46795
I20260807 15:55:07.320371 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:07.326409 9058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:07.326617 9059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:07.330060 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:07.330391 9062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:07.331394 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:07.331660 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:07.331861 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118107331830 us; error 0 us; skew 500 ppm
I20260807 15:55:07.332572 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:07.335407 5300 webserver.cc:460] Webserver started at http://127.5.45.62:43349/ using document root <none> and password file <none>
I20260807 15:55:07.336050 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:07.336292 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:07.336629 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:07.338193 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "9586fe15b7754f84b120ecbd02815d8d"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.344478 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 15:55:07.351958 9067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.352993 5300 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.006s
I20260807 15:55:07.353442 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "9586fe15b7754f84b120ecbd02815d8d"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.353895 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:07.375680 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:07.376757 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:07.425941 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:46795
I20260807 15:55:07.426105 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:46795 every 8 connection(s)
I20260807 15:55:07.430287 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:07.446615 9137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d: Bootstrap starting.
I20260807 15:55:07.450989 9137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:07.455842 9137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d: No bootstrap required, opened a new log
I20260807 15:55:07.458096 9137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER }
I20260807 15:55:07.458503 9137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:07.458758 9137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9586fe15b7754f84b120ecbd02815d8d, State: Initialized, Role: FOLLOWER
I20260807 15:55:07.459344 9137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER }
I20260807 15:55:07.459821 9137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:07.460105 9137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:07.460417 9137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:07.465226 9137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER }
I20260807 15:55:07.465931 9137 leader_election.cc:302] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9586fe15b7754f84b120ecbd02815d8d; no voters:
I20260807 15:55:07.467394 9137 leader_election.cc:288] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:07.467751 9141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:07.470613 9141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 1 LEADER]: Becoming Leader. State: Replica: 9586fe15b7754f84b120ecbd02815d8d, State: Running, Role: LEADER
I20260807 15:55:07.471455 9141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER }
I20260807 15:55:07.472265 9137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:07.474488 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9586fe15b7754f84b120ecbd02815d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER } }
I20260807 15:55:07.475067 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:07.476945 9144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9586fe15b7754f84b120ecbd02815d8d. Latest consensus state: current_term: 1 leader_uuid: "9586fe15b7754f84b120ecbd02815d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9586fe15b7754f84b120ecbd02815d8d" member_type: VOTER } }
I20260807 15:55:07.477715 9144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:07.486075 9148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:07.493199 9148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:07.495783 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:07.503299 9148 catalog_manager.cc:1347] Generated new cluster ID: fb0c60892a9e42feb1f196ef4f9dc717
I20260807 15:55:07.503670 9148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:07.568686 9148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:07.570492 9148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:07.594009 9148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d: Generated new TSK 0
I20260807 15:55:07.594744 9148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:07.627274 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:07.634636 9166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:07.636961 9167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:07.643769 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:07.644408 9169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:07.645692 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:07.645936 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:07.646112 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118107646092 us; error 0 us; skew 500 ppm
I20260807 15:55:07.646652 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:07.649806 5300 webserver.cc:460] Webserver started at http://127.5.45.1:34355/ using document root <none> and password file <none>
I20260807 15:55:07.650348 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:07.650575 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:07.650854 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:07.651988 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "13717eb3583d422ba78745ebceebf461"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.657279 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 15:55:07.661003 9175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.661921 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:55:07.662217 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "13717eb3583d422ba78745ebceebf461"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.662559 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:07.684065 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:07.685163 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:07.698917 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:07.699250 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.699541 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:07.699751 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.774523 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46203
I20260807 15:55:07.774622 9260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46203 every 8 connection(s)
I20260807 15:55:07.779619 9261 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46795
I20260807 15:55:07.780063 9261 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:07.780334 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:55:07.780927 9261 heartbeater.cc:511] Master 127.5.45.62:46795 requested a full tablet report, sending...
I20260807 15:55:07.783298 9088 ts_manager.cc:194] Registered new tserver with Master: 13717eb3583d422ba78745ebceebf461 (127.5.45.1:46203)
I20260807 15:55:07.786015 9088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46880
W20260807 15:55:07.787555 9263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:07.788733 9264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:07.792794 9268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:07.793265 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:07.794193 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:07.794425 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:07.794606 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118107794585 us; error 0 us; skew 500 ppm
I20260807 15:55:07.795122 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:07.798092 5300 webserver.cc:460] Webserver started at http://127.5.45.2:42383/ using document root <none> and password file <none>
I20260807 15:55:07.798581 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:07.798796 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:07.799072 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:07.800189 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "f2fe105a986042a1b10fe9424f586811"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.805037 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 15:55:07.808943 9274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.809921 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:55:07.810276 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "f2fe105a986042a1b10fe9424f586811"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:07.810634 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:07.824450 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:07.825871 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:07.832537 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:07.832825 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:07.833166 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:07.833389 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 15:55:07.909729 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:32811
I20260807 15:55:07.909827 9358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:32811 every 8 connection(s)
I20260807 15:55:07.915683 9359 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46795
I20260807 15:55:07.916177 9359 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:07.917109 9359 heartbeater.cc:511] Master 127.5.45.62:46795 requested a full tablet report, sending...
I20260807 15:55:07.919328 9088 ts_manager.cc:194] Registered new tserver with Master: f2fe105a986042a1b10fe9424f586811 (127.5.45.2:32811)
I20260807 15:55:07.920229 5300 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.00634462s
I20260807 15:55:07.921056 9088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46900
I20260807 15:55:07.963667 9219 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 15:55:07.965058 9219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:07.976696 9370 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Bootstrap starting.
I20260807 15:55:07.981179 9370 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:07.986593 9370 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: No bootstrap required, opened a new log
I20260807 15:55:07.987037 9370 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260807 15:55:07.989131 9370 raft_consensus.cc:348] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:07.989717 9370 raft_consensus.cc:374] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:07.989965 9370 raft_consensus.cc:729] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13717eb3583d422ba78745ebceebf461, State: Initialized, Role: FOLLOWER
I20260807 15:55:07.990545 9370 consensus_queue.cc:260] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:07.991060 9370 raft_consensus.cc:388] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:07.991314 9370 raft_consensus.cc:482] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:07.991590 9370 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:07.996732 9370 raft_consensus.cc:504] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:07.997453 9370 leader_election.cc:302] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13717eb3583d422ba78745ebceebf461; no voters:
I20260807 15:55:07.998764 9370 leader_election.cc:288] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:07.999151 9372 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:08.000653 9261 heartbeater.cc:503] Master 127.5.45.62:46795 was elected leader, sending a full tablet report...
I20260807 15:55:08.001592 9370 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260807 15:55:08.002116 9372 raft_consensus.cc:686] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 LEADER]: Becoming Leader. State: Replica: 13717eb3583d422ba78745ebceebf461, State: Running, Role: LEADER
I20260807 15:55:08.002916 9372 consensus_queue.cc:237] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.006104 9374 tablet_replica.cc:439] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: TxnStatusTablet state changed. Reason: New leader 13717eb3583d422ba78745ebceebf461. Latest consensus state: current_term: 1 leader_uuid: "13717eb3583d422ba78745ebceebf461" committed_config { opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } } }
I20260807 15:55:08.006740 9374 tablet_replica.cc:442] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:08.006924 9373 tablet_replica.cc:439] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13717eb3583d422ba78745ebceebf461" committed_config { opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } } }
I20260807 15:55:08.007507 9373 tablet_replica.cc:442] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: This TxnStatusTablet replica's current role is: LEADER
W20260807 15:55:08.010867 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:08.015175 9377 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:08.015905 9377 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 15:55:08.019049 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:08.078969 9382 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Initiating tablet copy from peer 13717eb3583d422ba78745ebceebf461 (127.5.45.1:46203)
I20260807 15:55:08.080922 9382 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: tablet copy: Beginning tablet copy session from remote peer at address 127.5.45.1:46203
I20260807 15:55:08.092432 9243 tablet_copy_service.cc:140] P 13717eb3583d422ba78745ebceebf461: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer f2fe105a986042a1b10fe9424f586811 ({username='slave'} at 127.0.0.1:54878)
I20260807 15:55:08.093011 9243 tablet_copy_service.cc:161] P 13717eb3583d422ba78745ebceebf461: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer f2fe105a986042a1b10fe9424f586811 at {username='slave'} at 127.0.0.1:54878: session id = f2fe105a986042a1b10fe9424f586811-11111111111111111111111111111111
I20260807 15:55:08.098712 9243 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:55:08.102226 9382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:08.115876 9382 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: tablet copy: Starting download of 0 data blocks...
I20260807 15:55:08.116459 9382 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: tablet copy: Starting download of 1 WAL segments...
I20260807 15:55:08.120385 9382 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:55:08.127281 9382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Bootstrap starting.
I20260807 15:55:08.150499 9382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:55:08.151309 9382 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Bootstrap complete.
I20260807 15:55:08.151911 9382 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.007s
I20260807 15:55:08.153947 9382 raft_consensus.cc:348] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.154390 9382 raft_consensus.cc:729] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: f2fe105a986042a1b10fe9424f586811, State: Initialized, Role: NON_PARTICIPANT
I20260807 15:55:08.154948 9382 consensus_queue.cc:260] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.156961 9385 tablet_replica.cc:439] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } } }
I20260807 15:55:08.157234 9359 heartbeater.cc:503] Master 127.5.45.62:46795 was elected leader, sending a full tablet report...
I20260807 15:55:08.157601 9385 tablet_replica.cc:442] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260807 15:55:08.157732 9382 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:55:08.159682 9243 tablet_copy_service.cc:342] P 13717eb3583d422ba78745ebceebf461: Request end of tablet copy session f2fe105a986042a1b10fe9424f586811-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:54878
I20260807 15:55:08.160087 9243 tablet_copy_service.cc:433] P 13717eb3583d422ba78745ebceebf461: ending tablet copy session f2fe105a986042a1b10fe9424f586811-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer f2fe105a986042a1b10fe9424f586811
W20260807 15:55:08.160575 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260807 15:55:08.165145 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:08.181383 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:08.212508 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:08.213774 5300 tablet_replica.cc:330] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: stopping tablet replica
I20260807 15:55:08.214321 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:08.214936 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:08.238241 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:08.250433 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:08.257166 9386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:08.260354 9387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:08.260496 9390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:08.262205 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:08.263401 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:08.263687 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:08.263878 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118108263866 us; error 0 us; skew 500 ppm
I20260807 15:55:08.264612 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:08.267567 5300 webserver.cc:460] Webserver started at http://127.5.45.1:34355/ using document root <none> and password file <none>
I20260807 15:55:08.268222 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:08.268438 5300 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260807 15:55:08.273597 5300 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 15:55:08.277446 9396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:08.278654 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:55:08.279000 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "13717eb3583d422ba78745ebceebf461"
format_stamp: "Formatted at 2026-08-07 15:55:07 on dist-test-slave-bw88"
I20260807 15:55:08.279299 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:08.291294 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:08.292523 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:08.300549 9405 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 15:55:08.309804 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:55:08.310225 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.001s sys 0.001s
I20260807 15:55:08.310642 5300 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:55:08.317258 5300 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:55:08.317569 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.007s sys 0.000s
I20260807 15:55:08.317989 9405 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Bootstrap starting.
I20260807 15:55:08.372301 9405 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:55:08.373473 9405 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Bootstrap complete.
I20260807 15:55:08.374817 9405 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Time spent bootstrapping tablet: real 0.057s user 0.031s sys 0.010s
I20260807 15:55:08.376845 9405 raft_consensus.cc:348] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.377315 9405 raft_consensus.cc:729] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13717eb3583d422ba78745ebceebf461, State: Initialized, Role: FOLLOWER
I20260807 15:55:08.378163 9405 consensus_queue.cc:260] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.378690 9405 raft_consensus.cc:388] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:08.378942 9405 raft_consensus.cc:482] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:08.379230 9405 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:55:08.384630 9405 raft_consensus.cc:504] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.385247 9405 leader_election.cc:302] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13717eb3583d422ba78745ebceebf461; no voters:
I20260807 15:55:08.388240 9405 leader_election.cc:288] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:55:08.388608 9443 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:55:08.402264 9443 raft_consensus.cc:686] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 2 LEADER]: Becoming Leader. State: Replica: 13717eb3583d422ba78745ebceebf461, State: Running, Role: LEADER
I20260807 15:55:08.403364 9443 consensus_queue.cc:237] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } }
I20260807 15:55:08.405009 9405 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461: Time spent starting tablet: real 0.030s user 0.016s sys 0.010s
I20260807 15:55:08.407613 9446 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 13717eb3583d422ba78745ebceebf461. Latest consensus state: current_term: 2 leader_uuid: "13717eb3583d422ba78745ebceebf461" committed_config { opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } } }
I20260807 15:55:08.408285 9446 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:08.409675 9447 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "13717eb3583d422ba78745ebceebf461" committed_config { opid_index: -1 peers { permanent_uuid: "13717eb3583d422ba78745ebceebf461" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 46203 } } }
I20260807 15:55:08.410321 9447 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:08.422710 9452 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:08.423318 9452 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:55:08.453512 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:46203
I20260807 15:55:08.453836 9485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:46203 every 8 connection(s)
I20260807 15:55:08.460820 9487 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46795
I20260807 15:55:08.461369 9487 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:08.461822 5300 tablet_server.cc:178] TabletServer@127.5.45.1:46203 shutting down...
I20260807 15:55:08.462358 9487 heartbeater.cc:511] Master 127.5.45.62:46795 requested a full tablet report, sending...
I20260807 15:55:08.469026 9088 ts_manager.cc:194] Re-registered known tserver with Master: 13717eb3583d422ba78745ebceebf461 (127.5.45.1:46203)
W20260807 15:55:08.469779 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:08.472056 9088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46946
I20260807 15:55:08.476397 9487 heartbeater.cc:503] Master 127.5.45.62:46795 was elected leader, sending a full tablet report...
W20260807 15:55:08.481631 9088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:08.493211 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:08.494149 5300 tablet_replica.cc:330] stopping tablet replica
I20260807 15:55:08.494709 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 2 LEADER]: Raft consensus shutting down.
I20260807 15:55:08.495177 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 13717eb3583d422ba78745ebceebf461 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:08.516408 5300 tablet_server.cc:194] TabletServer@127.5.45.1:46203 shutdown complete.
I20260807 15:55:08.528146 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:55:08.550676 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:08.551656 5300 tablet_replica.cc:330] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811: stopping tablet replica
I20260807 15:55:08.552356 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 15:55:08.552847 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P f2fe105a986042a1b10fe9424f586811 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 15:55:08.573998 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:55:08.588095 5300 master.cc:537] Master@127.5.45.62:46795 shutting down...
I20260807 15:55:08.616182 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:08.616750 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:08.617066 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9586fe15b7754f84b120ecbd02815d8d: stopping tablet replica
I20260807 15:55:08.635725 5300 master.cc:552] Master@127.5.45.62:46795 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1342 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260807 15:55:08.661701 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:35483
I20260807 15:55:08.662786 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:08.668316 9494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:08.668725 9495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:08.671779 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:08.671998 9499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:08.672991 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:08.673259 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:08.673467 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118108673451 us; error 0 us; skew 500 ppm
I20260807 15:55:08.674170 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:08.676971 5300 webserver.cc:460] Webserver started at http://127.5.45.62:37833/ using document root <none> and password file <none>
I20260807 15:55:08.677627 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:08.677862 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:08.678200 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:08.679663 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "6785589181a647589f56333371a9896c"
format_stamp: "Formatted at 2026-08-07 15:55:08 on dist-test-slave-bw88"
I20260807 15:55:08.685889 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 15:55:08.690316 9503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:08.691205 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:55:08.691530 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "6785589181a647589f56333371a9896c"
format_stamp: "Formatted at 2026-08-07 15:55:08 on dist-test-slave-bw88"
I20260807 15:55:08.691890 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:08.721565 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:08.723001 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:08.770642 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:35483
I20260807 15:55:08.770714 9566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:35483 every 8 connection(s)
I20260807 15:55:08.775548 9567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:08.785466 9567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c: Bootstrap starting.
I20260807 15:55:08.789746 9567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:08.794338 9567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c: No bootstrap required, opened a new log
I20260807 15:55:08.796314 9567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER }
I20260807 15:55:08.796716 9567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:08.796965 9567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6785589181a647589f56333371a9896c, State: Initialized, Role: FOLLOWER
I20260807 15:55:08.797597 9567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER }
I20260807 15:55:08.798043 9567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:08.798257 9567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:08.798537 9567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:08.803030 9567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER }
I20260807 15:55:08.803534 9567 leader_election.cc:302] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6785589181a647589f56333371a9896c; no voters:
I20260807 15:55:08.804912 9567 leader_election.cc:288] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:08.805274 9570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:08.807024 9570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 1 LEADER]: Becoming Leader. State: Replica: 6785589181a647589f56333371a9896c, State: Running, Role: LEADER
I20260807 15:55:08.807950 9570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER }
I20260807 15:55:08.808661 9567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:08.810897 9571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6785589181a647589f56333371a9896c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER } }
I20260807 15:55:08.810997 9573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6785589181a647589f56333371a9896c. Latest consensus state: current_term: 1 leader_uuid: "6785589181a647589f56333371a9896c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6785589181a647589f56333371a9896c" member_type: VOTER } }
I20260807 15:55:08.811506 9571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:08.811645 9573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:08.814795 9576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:08.823685 9576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:08.837616 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:08.837685 9576 catalog_manager.cc:1347] Generated new cluster ID: 39afe48088a044dc9cfb74cc2e896ddd
I20260807 15:55:08.838120 9576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:08.856084 9576 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:08.858531 9576 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:08.874893 9576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c: Generated new TSK 0
I20260807 15:55:08.875680 9576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:08.904889 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:08.911648 9593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:08.912793 9594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:08.915643 9596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:08.916278 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:08.917243 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:08.917513 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:08.917693 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118108917678 us; error 0 us; skew 500 ppm
I20260807 15:55:08.918233 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:08.921195 5300 webserver.cc:460] Webserver started at http://127.5.45.1:43763/ using document root <none> and password file <none>
I20260807 15:55:08.921733 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:08.921938 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:08.922207 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:08.923310 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "45c939a15c2846fd98c11932a3919197"
format_stamp: "Formatted at 2026-08-07 15:55:08 on dist-test-slave-bw88"
I20260807 15:55:08.928124 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:55:08.932056 9601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:08.932950 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:55:08.933249 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "45c939a15c2846fd98c11932a3919197"
format_stamp: "Formatted at 2026-08-07 15:55:08 on dist-test-slave-bw88"
I20260807 15:55:08.933555 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:08.967085 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:08.968297 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:08.981962 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:08.982290 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:08.982615 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:08.982854 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:09.072516 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:45207
I20260807 15:55:09.072865 9685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:45207 every 8 connection(s)
I20260807 15:55:09.078773 9686 heartbeater.cc:347] Connected to a master server at 127.5.45.62:35483
I20260807 15:55:09.079382 9686 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:09.080441 9686 heartbeater.cc:511] Master 127.5.45.62:35483 requested a full tablet report, sending...
I20260807 15:55:09.083137 9524 ts_manager.cc:194] Registered new tserver with Master: 45c939a15c2846fd98c11932a3919197 (127.5.45.1:45207)
I20260807 15:55:09.083827 5300 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004925204s
I20260807 15:55:09.086159 9524 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42576
I20260807 15:55:09.131556 9648 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 15:55:09.133158 9648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:09.150820 9699 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: Bootstrap starting.
I20260807 15:55:09.155241 9699 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:09.159822 9699 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: No bootstrap required, opened a new log
I20260807 15:55:09.160261 9699 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260807 15:55:09.162369 9699 raft_consensus.cc:348] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } }
I20260807 15:55:09.162850 9699 raft_consensus.cc:374] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:09.163100 9699 raft_consensus.cc:729] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c939a15c2846fd98c11932a3919197, State: Initialized, Role: FOLLOWER
I20260807 15:55:09.163733 9699 consensus_queue.cc:260] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } }
I20260807 15:55:09.164356 9699 raft_consensus.cc:388] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:09.164722 9699 raft_consensus.cc:482] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:09.165094 9699 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:09.171237 9699 raft_consensus.cc:504] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } }
I20260807 15:55:09.171870 9699 leader_election.cc:302] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45c939a15c2846fd98c11932a3919197; no voters:
I20260807 15:55:09.173399 9699 leader_election.cc:288] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:09.173918 9701 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:09.175799 9686 heartbeater.cc:503] Master 127.5.45.62:35483 was elected leader, sending a full tablet report...
I20260807 15:55:09.175992 9701 raft_consensus.cc:686] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 1 LEADER]: Becoming Leader. State: Replica: 45c939a15c2846fd98c11932a3919197, State: Running, Role: LEADER
I20260807 15:55:09.177175 9699 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: Time spent starting tablet: real 0.017s user 0.015s sys 0.002s
I20260807 15:55:09.176983 9701 consensus_queue.cc:237] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } }
I20260807 15:55:09.180598 9702 tablet_replica.cc:439] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45c939a15c2846fd98c11932a3919197" committed_config { opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } } }
I20260807 15:55:09.181241 9702 tablet_replica.cc:442] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:09.181170 9703 tablet_replica.cc:439] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: TxnStatusTablet state changed. Reason: New leader 45c939a15c2846fd98c11932a3919197. Latest consensus state: current_term: 1 leader_uuid: "45c939a15c2846fd98c11932a3919197" committed_config { opid_index: -1 peers { permanent_uuid: "45c939a15c2846fd98c11932a3919197" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 45207 } } }
I20260807 15:55:09.181964 9703 tablet_replica.cc:442] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: This TxnStatusTablet replica's current role is: LEADER
W20260807 15:55:09.187361 9524 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:09.190840 9705 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:09.191303 9705 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 15:55:09.193936 9524 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260807 15:55:09.248641 9638 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 15:55:09.249024 9638 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 15:55:09.249421 9638 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260807 15:55:09.258061 9638 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 15:55:09.301688 9638 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260807 15:55:09.302017 9638 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260807 15:55:09.310359 9638 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260807 15:55:09.328269 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.330623 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.332852 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.335222 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.337297 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.339502 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:34896
W20260807 15:55:09.352012 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
W20260807 15:55:09.354074 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
W20260807 15:55:09.355918 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
W20260807 15:55:09.357805 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
W20260807 15:55:09.359727 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
W20260807 15:55:09.361603 9648 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:34906
I20260807 15:55:09.367175 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:09.391077 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:09.391851 5300 tablet_replica.cc:330] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197: stopping tablet replica
I20260807 15:55:09.392392 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:09.392832 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 45c939a15c2846fd98c11932a3919197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:09.415465 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:09.427018 5300 master.cc:537] Master@127.5.45.62:35483 shutting down...
I20260807 15:55:09.443948 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:09.444515 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:09.444835 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6785589181a647589f56333371a9896c: stopping tablet replica
I20260807 15:55:09.469108 5300 master.cc:552] Master@127.5.45.62:35483 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (833 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260807 15:55:09.494825 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:39489
I20260807 15:55:09.495894 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:09.500912 9714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:09.502043 9716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:09.503969 9718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:09.504441 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:09.505187 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:09.505424 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:09.505578 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118109505562 us; error 0 us; skew 500 ppm
I20260807 15:55:09.506104 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:09.508567 5300 webserver.cc:460] Webserver started at http://127.5.45.62:37785/ using document root <none> and password file <none>
I20260807 15:55:09.509058 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:09.509292 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:09.509702 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:09.510851 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "b66fb69fc7154f7c8f4f7067a61b0264"
format_stamp: "Formatted at 2026-08-07 15:55:09 on dist-test-slave-bw88"
I20260807 15:55:09.515479 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:55:09.518937 9724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:09.519835 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:55:09.520090 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "b66fb69fc7154f7c8f4f7067a61b0264"
format_stamp: "Formatted at 2026-08-07 15:55:09 on dist-test-slave-bw88"
I20260807 15:55:09.520337 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:09.535233 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:09.536181 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:09.573222 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:39489
I20260807 15:55:09.573397 9790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:39489 every 8 connection(s)
I20260807 15:55:09.576998 9791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:09.587572 9791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264: Bootstrap starting.
I20260807 15:55:09.591863 9791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:09.596035 9791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264: No bootstrap required, opened a new log
I20260807 15:55:09.597990 9791 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER }
I20260807 15:55:09.598363 9791 raft_consensus.cc:374] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:09.598531 9791 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66fb69fc7154f7c8f4f7067a61b0264, State: Initialized, Role: FOLLOWER
I20260807 15:55:09.599099 9791 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER }
I20260807 15:55:09.599512 9791 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:09.599707 9791 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:09.599905 9791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:09.604332 9791 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER }
I20260807 15:55:09.604815 9791 leader_election.cc:302] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66fb69fc7154f7c8f4f7067a61b0264; no voters:
I20260807 15:55:09.605955 9791 leader_election.cc:288] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:09.606313 9794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:09.607683 9794 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 1 LEADER]: Becoming Leader. State: Replica: b66fb69fc7154f7c8f4f7067a61b0264, State: Running, Role: LEADER
I20260807 15:55:09.608387 9794 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER }
I20260807 15:55:09.609066 9791 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:09.612789 9795 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER } }
I20260807 15:55:09.612799 9798 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66fb69fc7154f7c8f4f7067a61b0264. Latest consensus state: current_term: 1 leader_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66fb69fc7154f7c8f4f7067a61b0264" member_type: VOTER } }
I20260807 15:55:09.613511 9795 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:09.613592 9798 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:09.619055 9801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:09.626698 9801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:09.627691 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:09.635182 9801 catalog_manager.cc:1347] Generated new cluster ID: 72649ead95114ad098a9ec0c7f59bb76
I20260807 15:55:09.635464 9801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:09.651300 9801 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:09.653015 9801 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:09.664660 9801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264: Generated new TSK 0
I20260807 15:55:09.665369 9801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:09.694679 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:09.700819 9816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:09.702153 9817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:09.704598 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:09.704873 9820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:09.705865 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:09.706045 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:09.706159 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118109706151 us; error 0 us; skew 500 ppm
I20260807 15:55:09.706682 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:09.709095 5300 webserver.cc:460] Webserver started at http://127.5.45.1:45581/ using document root <none> and password file <none>
I20260807 15:55:09.709681 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:09.709894 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:09.710208 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:09.711448 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "42fb6f06f89e4bf089bb67f000ebf046"
format_stamp: "Formatted at 2026-08-07 15:55:09 on dist-test-slave-bw88"
I20260807 15:55:09.716166 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:55:09.719448 9827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:09.720279 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:55:09.720543 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "42fb6f06f89e4bf089bb67f000ebf046"
format_stamp: "Formatted at 2026-08-07 15:55:09 on dist-test-slave-bw88"
I20260807 15:55:09.720870 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:09.736842 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:09.737880 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:09.743156 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:09.743490 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:09.743780 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:09.743988 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:09.801208 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:33753
I20260807 15:55:09.801331 9921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:33753 every 8 connection(s)
I20260807 15:55:09.805898 9922 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39489
I20260807 15:55:09.806278 9922 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:09.806994 9922 heartbeater.cc:511] Master 127.5.45.62:39489 requested a full tablet report, sending...
I20260807 15:55:09.808981 9746 ts_manager.cc:194] Registered new tserver with Master: 42fb6f06f89e4bf089bb67f000ebf046 (127.5.45.1:33753)
I20260807 15:55:09.809406 5300 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.005045989s
I20260807 15:55:09.811417 9746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55206
I20260807 15:55:09.845539 9871 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 15:55:09.846751 9871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:09.858534 9935 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: Bootstrap starting.
I20260807 15:55:09.863020 9935 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:09.867161 9935 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: No bootstrap required, opened a new log
I20260807 15:55:09.867506 9935 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 15:55:09.869256 9935 raft_consensus.cc:348] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.869735 9935 raft_consensus.cc:374] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:09.869915 9935 raft_consensus.cc:729] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fb6f06f89e4bf089bb67f000ebf046, State: Initialized, Role: FOLLOWER
I20260807 15:55:09.870527 9935 consensus_queue.cc:260] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.870980 9935 raft_consensus.cc:388] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:09.871173 9935 raft_consensus.cc:482] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:09.871371 9935 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:09.875916 9935 raft_consensus.cc:504] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.876397 9935 leader_election.cc:302] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42fb6f06f89e4bf089bb67f000ebf046; no voters:
I20260807 15:55:09.877521 9935 leader_election.cc:288] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:09.877902 9937 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:09.879359 9937 raft_consensus.cc:686] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 LEADER]: Becoming Leader. State: Replica: 42fb6f06f89e4bf089bb67f000ebf046, State: Running, Role: LEADER
I20260807 15:55:09.879544 9922 heartbeater.cc:503] Master 127.5.45.62:39489 was elected leader, sending a full tablet report...
I20260807 15:55:09.879793 9935 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260807 15:55:09.880136 9937 consensus_queue.cc:237] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.884265 9939 tablet_replica.cc:439] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: TxnStatusTablet state changed. Reason: New leader 42fb6f06f89e4bf089bb67f000ebf046. Latest consensus state: current_term: 1 leader_uuid: "42fb6f06f89e4bf089bb67f000ebf046" committed_config { opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } } }
I20260807 15:55:09.884794 9939 tablet_replica.cc:442] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:55:09.884164 9938 tablet_replica.cc:439] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42fb6f06f89e4bf089bb67f000ebf046" committed_config { opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } } }
I20260807 15:55:09.885555 9938 tablet_replica.cc:442] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: This TxnStatusTablet replica's current role is: LEADER
W20260807 15:55:09.887530 9746 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:09.891783 9942 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:55:09.892194 9942 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 15:55:09.893126 9746 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260807 15:55:09.919560 9871 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260807 15:55:09.920848 9871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:09.931511 9935 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: Bootstrap starting.
I20260807 15:55:09.935830 9935 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:09.940131 9935 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: No bootstrap required, opened a new log
I20260807 15:55:09.940575 9935 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 15:55:09.942579 9935 raft_consensus.cc:348] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.943048 9935 raft_consensus.cc:374] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:09.943292 9935 raft_consensus.cc:729] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fb6f06f89e4bf089bb67f000ebf046, State: Initialized, Role: FOLLOWER
I20260807 15:55:09.943799 9935 consensus_queue.cc:260] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.944298 9935 raft_consensus.cc:388] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:09.944525 9935 raft_consensus.cc:482] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:09.944787 9935 raft_consensus.cc:3037] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:09.949486 9935 raft_consensus.cc:504] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
I20260807 15:55:09.950042 9935 leader_election.cc:302] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42fb6f06f89e4bf089bb67f000ebf046; no voters:
I20260807 15:55:09.950547 9935 leader_election.cc:288] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:09.950770 9937 raft_consensus.cc:2781] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:09.951249 9937 raft_consensus.cc:686] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 LEADER]: Becoming Leader. State: Replica: 42fb6f06f89e4bf089bb67f000ebf046, State: Running, Role: LEADER
I20260807 15:55:09.952044 9935 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260807 15:55:09.951964 9937 consensus_queue.cc:237] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "42fb6f06f89e4bf089bb67f000ebf046" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 33753 } }
W20260807 15:55:09.957279 9746 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260807 15:55:09.961400 9746 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260807 15:55:09.982731 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:10.004442 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:10.005270 5300 tablet_replica.cc:330] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046: stopping tablet replica
I20260807 15:55:10.005942 5300 raft_consensus.cc:2229] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:10.006358 5300 raft_consensus.cc:2258] T 22222222222222222222222222222222 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:10.009159 5300 tablet_replica.cc:330] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046: stopping tablet replica
I20260807 15:55:10.009716 5300 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:10.010227 5300 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 42fb6f06f89e4bf089bb67f000ebf046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:10.032856 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:10.045842 5300 master.cc:537] Master@127.5.45.62:39489 shutting down...
I20260807 15:55:10.064369 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:10.064956 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:10.065456 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P b66fb69fc7154f7c8f4f7067a61b0264: stopping tablet replica
I20260807 15:55:10.075855 5300 master.cc:552] Master@127.5.45.62:39489 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (606 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3622 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260807 15:55:10.100765 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:46723
I20260807 15:55:10.101886 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:10.106904 9957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.108165 9958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.109377 9960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:10.110320 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:10.111099 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:10.111294 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:10.111454 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118110111425 us; error 0 us; skew 500 ppm
I20260807 15:55:10.111997 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:10.119678 5300 webserver.cc:460] Webserver started at http://127.5.45.62:40801/ using document root <none> and password file <none>
I20260807 15:55:10.120206 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:10.120409 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:10.120683 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:10.121913 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "86f4c52231f64c27a2d27b4c20989684"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.126332 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:55:10.129791 9965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.130563 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:55:10.130874 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "86f4c52231f64c27a2d27b4c20989684"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.131171 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:10.157171 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:10.158269 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:10.194612 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:46723
I20260807 15:55:10.194772 10032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:46723 every 8 connection(s)
I20260807 15:55:10.199003 10033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:10.210146 10033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684: Bootstrap starting.
I20260807 15:55:10.214792 10033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:10.219290 10033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684: No bootstrap required, opened a new log
I20260807 15:55:10.221236 10033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER }
I20260807 15:55:10.221673 10033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:10.221882 10033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f4c52231f64c27a2d27b4c20989684, State: Initialized, Role: FOLLOWER
I20260807 15:55:10.222538 10033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER }
I20260807 15:55:10.223029 10033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:10.223255 10033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:10.223522 10033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:10.228154 10033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER }
I20260807 15:55:10.228681 10033 leader_election.cc:302] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f4c52231f64c27a2d27b4c20989684; no voters:
I20260807 15:55:10.229959 10033 leader_election.cc:288] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:10.230324 10038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:10.232136 10038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 1 LEADER]: Becoming Leader. State: Replica: 86f4c52231f64c27a2d27b4c20989684, State: Running, Role: LEADER
I20260807 15:55:10.232880 10038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER }
I20260807 15:55:10.233724 10033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:10.235945 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f4c52231f64c27a2d27b4c20989684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER } }
I20260807 15:55:10.236753 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:10.235954 10040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f4c52231f64c27a2d27b4c20989684. Latest consensus state: current_term: 1 leader_uuid: "86f4c52231f64c27a2d27b4c20989684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f4c52231f64c27a2d27b4c20989684" member_type: VOTER } }
I20260807 15:55:10.237639 10040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:10.243602 10046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:10.248561 10046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:10.255343 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:10.258406 10046 catalog_manager.cc:1347] Generated new cluster ID: 11817f4d023a435297da1a96aa52d3bd
I20260807 15:55:10.258737 10046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:10.272585 10046 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:10.274299 10046 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:10.294797 10046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684: Generated new TSK 0
I20260807 15:55:10.295567 10046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:10.325824 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:10.332086 10062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.333576 10063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:10.335462 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:10.335911 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:10.336758 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:10.336978 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:10.337147 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118110337133 us; error 0 us; skew 500 ppm
I20260807 15:55:10.337719 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:10.340143 5300 webserver.cc:460] Webserver started at http://127.5.45.1:35069/ using document root <none> and password file <none>
I20260807 15:55:10.340631 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:10.340839 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:10.341110 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:10.342315 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "96f50c1df1b0434e9e65deeb0a76c4b2"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.346850 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:55:10.350433 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.351235 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:55:10.351532 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "96f50c1df1b0434e9e65deeb0a76c4b2"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.351845 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:10.372967 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:10.374095 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:10.375612 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:10.377969 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:10.378141 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.378422 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:10.378564 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.417982 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:40939
I20260807 15:55:10.418080 10153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:40939 every 8 connection(s)
I20260807 15:55:10.422606 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:10.430192 10158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.431233 10159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.433630 10161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:10.434882 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:10.436056 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:10.436327 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:10.436538 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118110436521 us; error 0 us; skew 500 ppm
I20260807 15:55:10.437130 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:10.437812 10154 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46723
I20260807 15:55:10.438167 10154 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:10.438833 10154 heartbeater.cc:511] Master 127.5.45.62:46723 requested a full tablet report, sending...
I20260807 15:55:10.440924 9989 ts_manager.cc:194] Registered new tserver with Master: 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939)
I20260807 15:55:10.441591 5300 webserver.cc:460] Webserver started at http://127.5.45.2:33605/ using document root <none> and password file <none>
I20260807 15:55:10.442112 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:10.442335 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:10.442619 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:10.443359 9989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37130
I20260807 15:55:10.443845 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "33e818790f194182a38af63938c7ffb6"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.448747 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:55:10.452147 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.453094 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:55:10.453406 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "33e818790f194182a38af63938c7ffb6"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.453686 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:10.472342 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:10.473483 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:10.474879 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:10.477192 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:10.477411 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 15:55:10.477671 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:10.477821 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.516659 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:35777
I20260807 15:55:10.516779 10244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:35777 every 8 connection(s)
I20260807 15:55:10.521456 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:10.528976 10248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.531394 10249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:10.533430 10251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:10.533600 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:10.534673 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:10.534986 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:10.535205 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118110535187 us; error 0 us; skew 500 ppm
I20260807 15:55:10.535887 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:10.536481 10245 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46723
I20260807 15:55:10.536933 10245 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:10.537839 10245 heartbeater.cc:511] Master 127.5.45.62:46723 requested a full tablet report, sending...
I20260807 15:55:10.538794 5300 webserver.cc:460] Webserver started at http://127.5.45.3:43515/ using document root <none> and password file <none>
I20260807 15:55:10.539330 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:10.539541 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:10.539858 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:10.540040 9989 ts_manager.cc:194] Registered new tserver with Master: 33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777)
I20260807 15:55:10.541059 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-2-root/instance:
uuid: "ecb4269cfe494dcd8d0f090e96539a8b"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.541841 9989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37140
I20260807 15:55:10.546169 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:55:10.549889 10257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.550873 5300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:55:10.551167 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-2-root
uuid: "ecb4269cfe494dcd8d0f090e96539a8b"
format_stamp: "Formatted at 2026-08-07 15:55:10 on dist-test-slave-bw88"
I20260807 15:55:10.551415 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:10.574476 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:10.575644 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:10.577116 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:10.579447 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:10.579630 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.579938 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:10.580080 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:10.622941 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.3:34695
I20260807 15:55:10.623080 10336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.3:34695 every 8 connection(s)
I20260807 15:55:10.678562 10338 heartbeater.cc:347] Connected to a master server at 127.5.45.62:46723
I20260807 15:55:10.679343 10338 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:10.680326 10338 heartbeater.cc:511] Master 127.5.45.62:46723 requested a full tablet report, sending...
I20260807 15:55:10.682924 9989 ts_manager.cc:194] Registered new tserver with Master: ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695)
I20260807 15:55:10.683741 5300 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.057492655s
I20260807 15:55:10.685318 9989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37144
I20260807 15:55:10.711163 5300 test_util.cc:268] Using random seed: 126094684
I20260807 15:55:10.738425 9989 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37174:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:55:10.799659 10113 tablet_service.cc:1467] Processing CreateTablet for tablet a632d1c1c21e4a84beca5042d35198f4 (DEFAULT_TABLE table=test-workload [id=c97d194917514a91b8ed6c25891f0081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:10.799690 10205 tablet_service.cc:1467] Processing CreateTablet for tablet a632d1c1c21e4a84beca5042d35198f4 (DEFAULT_TABLE table=test-workload [id=c97d194917514a91b8ed6c25891f0081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:10.801438 10113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a632d1c1c21e4a84beca5042d35198f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:10.801466 10205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a632d1c1c21e4a84beca5042d35198f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:10.804136 10294 tablet_service.cc:1467] Processing CreateTablet for tablet a632d1c1c21e4a84beca5042d35198f4 (DEFAULT_TABLE table=test-workload [id=c97d194917514a91b8ed6c25891f0081]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:10.805801 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a632d1c1c21e4a84beca5042d35198f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:10.818652 10363 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Bootstrap starting.
I20260807 15:55:10.821774 10364 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Bootstrap starting.
I20260807 15:55:10.828534 10363 tablet_bootstrap.cc:654] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:10.828907 10364 tablet_bootstrap.cc:654] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:10.830438 10365 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: Bootstrap starting.
I20260807 15:55:10.837402 10363 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: No bootstrap required, opened a new log
I20260807 15:55:10.838016 10364 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: No bootstrap required, opened a new log
I20260807 15:55:10.838090 10365 tablet_bootstrap.cc:654] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:10.838008 10363 ts_tablet_manager.cc:1397] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.011s
I20260807 15:55:10.838526 10364 ts_tablet_manager.cc:1397] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20260807 15:55:10.841033 10364 raft_consensus.cc:348] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.842247 10364 raft_consensus.cc:374] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:10.841717 10363 raft_consensus.cc:348] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.842711 10364 raft_consensus.cc:729] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e818790f194182a38af63938c7ffb6, State: Initialized, Role: FOLLOWER
I20260807 15:55:10.842825 10363 raft_consensus.cc:374] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:10.843290 10363 raft_consensus.cc:729] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f50c1df1b0434e9e65deeb0a76c4b2, State: Initialized, Role: FOLLOWER
I20260807 15:55:10.844074 10363 consensus_queue.cc:260] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.843757 10364 consensus_queue.cc:260] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.845057 10365 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: No bootstrap required, opened a new log
I20260807 15:55:10.845582 10365 ts_tablet_manager.cc:1397] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.007s
I20260807 15:55:10.847698 10364 ts_tablet_manager.cc:1428] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Time spent starting tablet: real 0.009s user 0.002s sys 0.006s
I20260807 15:55:10.847898 10154 heartbeater.cc:503] Master 127.5.45.62:46723 was elected leader, sending a full tablet report...
I20260807 15:55:10.848047 10363 ts_tablet_manager.cc:1428] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Time spent starting tablet: real 0.009s user 0.001s sys 0.008s
I20260807 15:55:10.849218 10245 heartbeater.cc:503] Master 127.5.45.62:46723 was elected leader, sending a full tablet report...
I20260807 15:55:10.848879 10365 raft_consensus.cc:348] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.850497 10365 raft_consensus.cc:374] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:10.850805 10365 raft_consensus.cc:729] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb4269cfe494dcd8d0f090e96539a8b, State: Initialized, Role: FOLLOWER
I20260807 15:55:10.851747 10365 consensus_queue.cc:260] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:10.854616 10338 heartbeater.cc:503] Master 127.5.45.62:46723 was elected leader, sending a full tablet report...
I20260807 15:55:10.855062 10365 ts_tablet_manager.cc:1428] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20260807 15:55:11.074947 10372 raft_consensus.cc:482] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:55:11.075525 10372 raft_consensus.cc:504] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:11.077793 10372 leader_election.cc:288] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939), ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695)
I20260807 15:55:11.097095 10125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a632d1c1c21e4a84beca5042d35198f4" candidate_uuid: "33e818790f194182a38af63938c7ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" is_pre_election: true
I20260807 15:55:11.097517 10306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a632d1c1c21e4a84beca5042d35198f4" candidate_uuid: "33e818790f194182a38af63938c7ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" is_pre_election: true
I20260807 15:55:11.098052 10125 raft_consensus.cc:2454] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e818790f194182a38af63938c7ffb6 in term 0.
I20260807 15:55:11.098174 10306 raft_consensus.cc:2454] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e818790f194182a38af63938c7ffb6 in term 0.
I20260807 15:55:11.099350 10173 leader_election.cc:302] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e818790f194182a38af63938c7ffb6, 96f50c1df1b0434e9e65deeb0a76c4b2; no voters:
I20260807 15:55:11.100096 10372 raft_consensus.cc:2781] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:55:11.100466 10372 raft_consensus.cc:482] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:55:11.100754 10372 raft_consensus.cc:3037] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:11.106355 10372 raft_consensus.cc:504] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:11.107959 10372 leader_election.cc:288] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [CANDIDATE]: Term 1 election: Requested vote from peers 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939), ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695)
I20260807 15:55:11.108857 10125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a632d1c1c21e4a84beca5042d35198f4" candidate_uuid: "33e818790f194182a38af63938c7ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2"
I20260807 15:55:11.109105 10306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a632d1c1c21e4a84beca5042d35198f4" candidate_uuid: "33e818790f194182a38af63938c7ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb4269cfe494dcd8d0f090e96539a8b"
I20260807 15:55:11.109411 10125 raft_consensus.cc:3037] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:11.109762 10306 raft_consensus.cc:3037] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:11.115063 10125 raft_consensus.cc:2454] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e818790f194182a38af63938c7ffb6 in term 1.
I20260807 15:55:11.115366 10306 raft_consensus.cc:2454] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e818790f194182a38af63938c7ffb6 in term 1.
I20260807 15:55:11.116110 10173 leader_election.cc:302] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e818790f194182a38af63938c7ffb6, 96f50c1df1b0434e9e65deeb0a76c4b2; no voters:
I20260807 15:55:11.116926 10372 raft_consensus.cc:2781] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:11.117893 10372 raft_consensus.cc:686] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Becoming Leader. State: Replica: 33e818790f194182a38af63938c7ffb6, State: Running, Role: LEADER
I20260807 15:55:11.118589 10372 consensus_queue.cc:237] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:11.125662 9989 catalog_manager.cc:5495] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33e818790f194182a38af63938c7ffb6 (127.5.45.2). New cstate: current_term: 1 leader_uuid: "33e818790f194182a38af63938c7ffb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:11.419760 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:11.419986 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:11.420812 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:11.584189 10372 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:55:11.601751 10372 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a632d1c1c21e4a84beca5042d35198f4 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): RUNNING
33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777): RUNNING [LEADER]
ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695): RUNNING
All reported replicas are:
A = 96f50c1df1b0434e9e65deeb0a76c4b2
B = 33e818790f194182a38af63938c7ffb6
C = ecb4269cfe494dcd8d0f090e96539a8b
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:11.665180 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:12.040716 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:12.041791 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:12.043061 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 15:55:12.257447 5300 log_verifier.cc:126] Checking tablet a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:12.324792 5300 log_verifier.cc:177] Verified matching terms for 1 ops in tablet a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:12.325456 5300 tablet_replica.cc:330] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: stopping tablet replica
I20260807 15:55:12.326004 5300 raft_consensus.cc:2229] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:12.326519 5300 raft_consensus.cc:2258] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:12.555727 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:12.557754 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:12.560921 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20260807 15:55:12.580760 10173 consensus_peers.cc:487] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 -> Peer 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): Couldn't send request to peer 96f50c1df1b0434e9e65deeb0a76c4b2. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:12.582263 10460 raft_consensus.cc:998] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Attempting to remove follower 96f50c1df1b0434e9e65deeb0a76c4b2 from the Raft config. Reason: The tablet replica hosted on peer 96f50c1df1b0434e9e65deeb0a76c4b2 has failed
I20260807 15:55:12.584810 10460 consensus_queue.cc:237] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:12.591689 10305 raft_consensus.cc:1261] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Refusing update from remote peer 33e818790f194182a38af63938c7ffb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:55:12.593384 10498 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:55:12.601702 10460 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:12.613904 10305 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:12.630460 9988 catalog_manager.cc:5495] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 reported cstate change: config changed from index -1 to 2, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New cstate: current_term: 1 leader_uuid: "33e818790f194182a38af63938c7ffb6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:12.693995 10218 consensus_queue.cc:237] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } }
I20260807 15:55:12.695540 10113 tablet_service.cc:1514] Processing DeleteTablet for tablet a632d1c1c21e4a84beca5042d35198f4 with delete_type TABLET_DATA_TOMBSTONED (TS 96f50c1df1b0434e9e65deeb0a76c4b2 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:42064
W20260807 15:55:12.707281 10173 consensus_peers.cc:487] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 -> Peer 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): Couldn't send request to peer 96f50c1df1b0434e9e65deeb0a76c4b2. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:12.708328 10505 ts_tablet_manager.cc:1905] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 15:55:12.710486 10305 raft_consensus.cc:1261] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Refusing update from remote peer 33e818790f194182a38af63938c7ffb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 15:55:12.712101 10460 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 15:55:12.719833 10498 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
I20260807 15:55:12.726634 10505 ts_tablet_manager.cc:1918] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260807 15:55:12.727154 10505 log.cc:1196] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786118049188364-5300-0/minicluster-data/ts-0-root/wals/a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:12.729532 9972 catalog_manager.cc:4841] TS 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): tablet a632d1c1c21e4a84beca5042d35198f4 (table test-workload [id=c97d194917514a91b8ed6c25891f0081]) successfully deleted
I20260807 15:55:12.728451 10306 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
I20260807 15:55:12.737720 9967 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet a632d1c1c21e4a84beca5042d35198f4 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260807 15:55:12.747586 9988 catalog_manager.cc:5495] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b reported cstate change: config changed from index 2 to 3, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New cstate: current_term: 1 leader_uuid: "33e818790f194182a38af63938c7ffb6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a632d1c1c21e4a84beca5042d35198f4 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777): RUNNING [LEADER]
ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695): RUNNING
All reported replicas are:
A = 96f50c1df1b0434e9e65deeb0a76c4b2
B = 33e818790f194182a38af63938c7ffb6
C = ecb4269cfe494dcd8d0f090e96539a8b
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | [config not available] | | |
B | A B* C | 1 | -1 | Yes
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:12.829850 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:13.221252 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:13.227556 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:13.232890 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:13.311781 10544 raft_consensus.cc:998] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Attempting to remove follower 96f50c1df1b0434e9e65deeb0a76c4b2 from the Raft config. Reason: The tablet replica hosted on peer 96f50c1df1b0434e9e65deeb0a76c4b2 has failed
I20260807 15:55:13.313477 10544 consensus_queue.cc:237] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } }
I20260807 15:55:13.319209 10306 raft_consensus.cc:1261] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Refusing update from remote peer 33e818790f194182a38af63938c7ffb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 15:55:13.320899 10540 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 15:55:13.329677 10306 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:13.329706 10541 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:13.343366 9989 catalog_manager.cc:5495] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 reported cstate change: config changed from index 3 to 4, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) evicted. New cstate: current_term: 1 leader_uuid: "33e818790f194182a38af63938c7ffb6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } }
I20260807 15:55:13.370618 10113 tablet_service.cc:1514] Processing DeleteTablet for tablet a632d1c1c21e4a84beca5042d35198f4 with delete_type TABLET_DATA_TOMBSTONED (TS 96f50c1df1b0434e9e65deeb0a76c4b2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42064
I20260807 15:55:13.376026 10218 consensus_queue.cc:237] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } }
I20260807 15:55:13.377899 9972 catalog_manager.cc:4841] TS 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): tablet a632d1c1c21e4a84beca5042d35198f4 (table test-workload [id=c97d194917514a91b8ed6c25891f0081]) successfully deleted
I20260807 15:55:13.382716 10306 raft_consensus.cc:1261] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Refusing update from remote peer 33e818790f194182a38af63938c7ffb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260807 15:55:13.383138 10173 consensus_peers.cc:487] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 -> Peer 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): Couldn't send request to peer 96f50c1df1b0434e9e65deeb0a76c4b2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:13.384135 10541 consensus_queue.cc:1035] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 15:55:13.391101 10545 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
I20260807 15:55:13.392729 10306 raft_consensus.cc:2932] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
I20260807 15:55:13.402062 9967 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet a632d1c1c21e4a84beca5042d35198f4 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260807 15:55:13.405560 9988 catalog_manager.cc:5495] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 reported cstate change: config changed from index 4 to 5, VOTER 96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1) added. New cstate: current_term: 1 leader_uuid: "33e818790f194182a38af63938c7ffb6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } } }
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a632d1c1c21e4a84beca5042d35198f4 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777): RUNNING [LEADER]
ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695): RUNNING
96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = 33e818790f194182a38af63938c7ffb6
B = ecb4269cfe494dcd8d0f090e96539a8b
C = 96f50c1df1b0434e9e65deeb0a76c4b2
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 3 | Yes
B | A* B C | 1 | 3 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:13.468523 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:13.772419 10560 ts_tablet_manager.cc:927] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Initiating tablet copy from peer 33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777)
I20260807 15:55:13.775758 10560 tablet_copy_client.cc:316] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet copy: Beginning tablet copy session from remote peer at address 127.5.45.2:35777
I20260807 15:55:13.790428 10228 tablet_copy_service.cc:140] P 33e818790f194182a38af63938c7ffb6: Received BeginTabletCopySession request for tablet a632d1c1c21e4a84beca5042d35198f4 from peer 96f50c1df1b0434e9e65deeb0a76c4b2 ({username='slave'} at 127.0.0.1:53990)
I20260807 15:55:13.790978 10228 tablet_copy_service.cc:161] P 33e818790f194182a38af63938c7ffb6: Beginning new tablet copy session on tablet a632d1c1c21e4a84beca5042d35198f4 from peer 96f50c1df1b0434e9e65deeb0a76c4b2 at {username='slave'} at 127.0.0.1:53990: session id = 96f50c1df1b0434e9e65deeb0a76c4b2-a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:13.797554 10228 tablet_copy_source_session.cc:205] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:55:13.799873 10560 ts_tablet_manager.cc:1905] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260807 15:55:13.811281 10560 ts_tablet_manager.cc:1918] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260807 15:55:13.812143 10560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a632d1c1c21e4a84beca5042d35198f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:13.818126 10560 tablet_copy_client.cc:791] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet copy: Starting download of 0 data blocks...
I20260807 15:55:13.818764 10560 tablet_copy_client.cc:655] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet copy: Starting download of 1 WAL segments...
I20260807 15:55:13.822476 10560 tablet_copy_client.cc:531] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:55:13.829032 10560 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Bootstrap starting.
I20260807 15:55:13.854465 10560 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:55:13.855615 10560 tablet_bootstrap.cc:492] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Bootstrap complete.
I20260807 15:55:13.856463 10560 ts_tablet_manager.cc:1397] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Time spent bootstrapping tablet: real 0.028s user 0.023s sys 0.004s
I20260807 15:55:13.859189 10560 raft_consensus.cc:348] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } }
I20260807 15:55:13.860037 10560 raft_consensus.cc:729] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f50c1df1b0434e9e65deeb0a76c4b2, State: Initialized, Role: FOLLOWER
I20260807 15:55:13.860791 10560 consensus_queue.cc:260] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "33e818790f194182a38af63938c7ffb6" member_type: VOTER last_known_addr { host: "127.5.45.2" port: 35777 } } peers { permanent_uuid: "ecb4269cfe494dcd8d0f090e96539a8b" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 34695 } } peers { permanent_uuid: "96f50c1df1b0434e9e65deeb0a76c4b2" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 40939 } }
I20260807 15:55:13.863446 10560 ts_tablet_manager.cc:1428] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260807 15:55:13.865042 10228 tablet_copy_service.cc:342] P 33e818790f194182a38af63938c7ffb6: Request end of tablet copy session 96f50c1df1b0434e9e65deeb0a76c4b2-a632d1c1c21e4a84beca5042d35198f4 received from {username='slave'} at 127.0.0.1:53990
I20260807 15:55:13.865415 10228 tablet_copy_service.cc:433] P 33e818790f194182a38af63938c7ffb6: ending tablet copy session 96f50c1df1b0434e9e65deeb0a76c4b2-a632d1c1c21e4a84beca5042d35198f4 on tablet a632d1c1c21e4a84beca5042d35198f4 with peer 96f50c1df1b0434e9e65deeb0a76c4b2
I20260807 15:55:13.934114 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:13.936619 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:13.939909 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a632d1c1c21e4a84beca5042d35198f4 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
33e818790f194182a38af63938c7ffb6 (127.5.45.2:35777): RUNNING [LEADER]
ecb4269cfe494dcd8d0f090e96539a8b (127.5.45.3:34695): RUNNING
96f50c1df1b0434e9e65deeb0a76c4b2 (127.5.45.1:40939): RUNNING
All reported replicas are:
A = 33e818790f194182a38af63938c7ffb6
B = ecb4269cfe494dcd8d0f090e96539a8b
C = 96f50c1df1b0434e9e65deeb0a76c4b2
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 5 | Yes
B | A* B C | 1 | 5 | Yes
C | A B C | 1 | 5 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:14.140470 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:14.366639 10125 raft_consensus.cc:1203] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260807 15:55:14.705513 10205 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:14.721138 10113 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:14.721738 10294 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
86f4c52231f64c27a2d27b4c20989684 | 127.5.45.62:46723 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
33e818790f194182a38af63938c7ffb6 | 127.5.45.2:35777 | HEALTHY | <none> | 1 | 0
96f50c1df1b0434e9e65deeb0a76c4b2 | 127.5.45.1:40939 | HEALTHY | <none> | 0 | 0
ecb4269cfe494dcd8d0f090e96539a8b | 127.5.45.3:34695 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 15:55:14.911056 5300 log_verifier.cc:126] Checking tablet a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:14.973587 5300 log_verifier.cc:177] Verified matching terms for 5 ops in tablet a632d1c1c21e4a84beca5042d35198f4
I20260807 15:55:14.976349 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:15.006614 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:15.007397 5300 tablet_replica.cc:330] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2: stopping tablet replica
I20260807 15:55:15.007974 5300 raft_consensus.cc:2229] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:15.008404 5300 raft_consensus.cc:2258] T a632d1c1c21e4a84beca5042d35198f4 P 96f50c1df1b0434e9e65deeb0a76c4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:15.029592 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:15.040799 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:55:15.066509 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:15.067255 5300 tablet_replica.cc:330] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6: stopping tablet replica
I20260807 15:55:15.067840 5300 raft_consensus.cc:2229] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:15.068692 5300 raft_consensus.cc:2258] T a632d1c1c21e4a84beca5042d35198f4 P 33e818790f194182a38af63938c7ffb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:15.090157 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:55:15.102521 5300 tablet_server.cc:178] TabletServer@127.5.45.3:0 shutting down...
I20260807 15:55:15.127696 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:15.128381 5300 tablet_replica.cc:330] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b: stopping tablet replica
I20260807 15:55:15.128930 5300 raft_consensus.cc:2229] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:15.129595 5300 raft_consensus.cc:2258] T a632d1c1c21e4a84beca5042d35198f4 P ecb4269cfe494dcd8d0f090e96539a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:15.150717 5300 tablet_server.cc:194] TabletServer@127.5.45.3:0 shutdown complete.
I20260807 15:55:15.161427 5300 master.cc:537] Master@127.5.45.62:46723 shutting down...
I20260807 15:55:15.183431 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:15.184005 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:15.184386 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 86f4c52231f64c27a2d27b4c20989684: stopping tablet replica
I20260807 15:55:15.202566 5300 master.cc:552] Master@127.5.45.62:46723 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (5135 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260807 15:55:15.237015 5300 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.45.62:39233
I20260807 15:55:15.238157 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:15.243741 10626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.244760 10627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.247787 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:15.248091 10629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.248986 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:15.249176 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:15.249408 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118115249391 us; error 0 us; skew 500 ppm
I20260807 15:55:15.249938 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:15.255256 5300 webserver.cc:460] Webserver started at http://127.5.45.62:44521/ using document root <none> and password file <none>
I20260807 15:55:15.255760 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:15.255954 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:15.256240 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:15.257460 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/master-0-root/instance:
uuid: "59d7c6067c16480c9c4a4855ad7c728f"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.262092 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 15:55:15.265522 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.266263 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:55:15.266566 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/master-0-root
uuid: "59d7c6067c16480c9c4a4855ad7c728f"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.266865 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:15.279469 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:15.280736 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:15.321805 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.62:39233
I20260807 15:55:15.321914 10700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.62:39233 every 8 connection(s)
I20260807 15:55:15.325719 10701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:15.337102 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f: Bootstrap starting.
I20260807 15:55:15.341789 10701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:15.346261 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f: No bootstrap required, opened a new log
I20260807 15:55:15.348332 10701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER }
I20260807 15:55:15.348771 10701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:15.349026 10701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d7c6067c16480c9c4a4855ad7c728f, State: Initialized, Role: FOLLOWER
I20260807 15:55:15.349702 10701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER }
I20260807 15:55:15.350157 10701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:55:15.350385 10701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:55:15.350663 10701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:15.355516 10701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER }
I20260807 15:55:15.356055 10701 leader_election.cc:302] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59d7c6067c16480c9c4a4855ad7c728f; no voters:
I20260807 15:55:15.357259 10701 leader_election.cc:288] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:55:15.357795 10704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:15.359179 10704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 1 LEADER]: Becoming Leader. State: Replica: 59d7c6067c16480c9c4a4855ad7c728f, State: Running, Role: LEADER
I20260807 15:55:15.359889 10704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER }
I20260807 15:55:15.360404 10701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:55:15.362902 10705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59d7c6067c16480c9c4a4855ad7c728f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER } }
I20260807 15:55:15.362941 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59d7c6067c16480c9c4a4855ad7c728f. Latest consensus state: current_term: 1 leader_uuid: "59d7c6067c16480c9c4a4855ad7c728f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d7c6067c16480c9c4a4855ad7c728f" member_type: VOTER } }
I20260807 15:55:15.363636 10705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:15.363757 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [sys.catalog]: This master's current role is: LEADER
I20260807 15:55:15.369515 10710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:55:15.377526 10710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:55:15.379233 5300 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 15:55:15.386559 10710 catalog_manager.cc:1347] Generated new cluster ID: a26b0a5df67042129d62decdd4db2333
I20260807 15:55:15.386857 10710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:55:15.406628 10710 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:55:15.408416 10710 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:55:15.423048 10710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f: Generated new TSK 0
I20260807 15:55:15.423731 10710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:55:15.445976 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:15.452163 10723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.453418 10724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.455577 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:15.456054 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.457118 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:15.457309 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:15.457541 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118115457506 us; error 0 us; skew 500 ppm
I20260807 15:55:15.458074 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:15.460587 5300 webserver.cc:460] Webserver started at http://127.5.45.1:42527/ using document root <none> and password file <none>
I20260807 15:55:15.461148 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:15.461452 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:15.461807 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:15.462950 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root/instance:
uuid: "70f1825e113848c5975ccfc9561809b5"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.467414 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:55:15.470894 10734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.471684 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:55:15.471972 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root
uuid: "70f1825e113848c5975ccfc9561809b5"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.472254 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:15.487624 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:15.488796 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:15.490429 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:15.492700 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:15.492913 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.493163 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:15.493328 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.535109 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.1:38043
I20260807 15:55:15.535197 10815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.1:38043 every 8 connection(s)
I20260807 15:55:15.539786 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:15.547583 10821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.549738 10822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.551582 5300 server_base.cc:1034] running on GCE node
W20260807 15:55:15.552644 10825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.553716 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:15.554004 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:15.554234 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118115554213 us; error 0 us; skew 500 ppm
I20260807 15:55:15.554940 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:15.555547 10816 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39233
I20260807 15:55:15.556018 10816 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:15.556854 10816 heartbeater.cc:511] Master 127.5.45.62:39233 requested a full tablet report, sending...
I20260807 15:55:15.557838 5300 webserver.cc:460] Webserver started at http://127.5.45.2:45485/ using document root <none> and password file <none>
I20260807 15:55:15.558473 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:15.558712 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:15.559022 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:15.559180 10654 ts_manager.cc:194] Registered new tserver with Master: 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043)
I20260807 15:55:15.560528 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-1-root/instance:
uuid: "1c21870d2b0946a4b0718ce8d4f49106"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.562125 10654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42202
I20260807 15:55:15.565828 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 15:55:15.569259 10830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.570091 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:55:15.570379 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-1-root
uuid: "1c21870d2b0946a4b0718ce8d4f49106"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.570660 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:15.584177 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:15.585390 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:15.586850 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:15.589181 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:15.589430 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 15:55:15.589674 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:15.589839 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.629925 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.2:36669
I20260807 15:55:15.630055 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.2:36669 every 8 connection(s)
I20260807 15:55:15.634781 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:15.642390 10920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.643200 10921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.646522 10924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.647598 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:15.648761 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:15.649056 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:15.649268 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118115649249 us; error 0 us; skew 500 ppm
I20260807 15:55:15.650005 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:15.652359 10916 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39233
I20260807 15:55:15.652755 10916 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:15.653456 5300 webserver.cc:460] Webserver started at http://127.5.45.3:36613/ using document root <none> and password file <none>
I20260807 15:55:15.653688 10916 heartbeater.cc:511] Master 127.5.45.62:39233 requested a full tablet report, sending...
I20260807 15:55:15.653928 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:15.654145 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:15.654358 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:15.655557 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-2-root/instance:
uuid: "87f6d897fceb4200bb49b3bd0dde5620"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.655941 10654 ts_manager.cc:194] Registered new tserver with Master: 1c21870d2b0946a4b0718ce8d4f49106 (127.5.45.2:36669)
I20260807 15:55:15.657660 10654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42212
I20260807 15:55:15.660537 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:55:15.664085 10930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.664846 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:55:15.665143 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-2-root
uuid: "87f6d897fceb4200bb49b3bd0dde5620"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.665494 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:15.675673 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:15.676788 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:15.678334 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:15.680511 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:15.680693 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.680874 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:15.680994 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.720088 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.3:35639
I20260807 15:55:15.720197 11023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.3:35639 every 8 connection(s)
I20260807 15:55:15.725165 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:55:15.732079 11028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.734961 11029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:55:15.736271 11031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:55:15.736860 5300 server_base.cc:1034] running on GCE node
I20260807 15:55:15.737815 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 15:55:15.738101 5300 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 15:55:15.738312 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786118115738294 us; error 0 us; skew 500 ppm
I20260807 15:55:15.738875 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 15:55:15.739539 11024 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39233
I20260807 15:55:15.739959 11024 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:15.740737 11024 heartbeater.cc:511] Master 127.5.45.62:39233 requested a full tablet report, sending...
I20260807 15:55:15.741640 5300 webserver.cc:460] Webserver started at http://127.5.45.4:35049/ using document root <none> and password file <none>
I20260807 15:55:15.742283 5300 fs_manager.cc:359] Metadata directory not provided
I20260807 15:55:15.742511 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:55:15.742842 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:55:15.742898 10654 ts_manager.cc:194] Registered new tserver with Master: 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639)
I20260807 15:55:15.743994 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-3-root/instance:
uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.744670 10654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42222
I20260807 15:55:15.748618 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 15:55:15.752113 11037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.752887 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:55:15.753199 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-3-root
uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5"
format_stamp: "Formatted at 2026-08-07 15:55:15 on dist-test-slave-bw88"
I20260807 15:55:15.753540 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:55:15.790413 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:55:15.791649 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:55:15.792949 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:55:15.795348 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:55:15.795581 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.795825 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:55:15.795991 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:55:15.837647 5300 rpc_server.cc:307] RPC server started. Bound to: 127.5.45.4:41677
I20260807 15:55:15.837751 11117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.45.4:41677 every 8 connection(s)
I20260807 15:55:15.853297 11118 heartbeater.cc:347] Connected to a master server at 127.5.45.62:39233
I20260807 15:55:15.853781 11118 heartbeater.cc:464] Registering TS with master...
I20260807 15:55:15.854547 11118 heartbeater.cc:511] Master 127.5.45.62:39233 requested a full tablet report, sending...
I20260807 15:55:15.856523 10654 ts_manager.cc:194] Registered new tserver with Master: e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677)
I20260807 15:55:15.857093 5300 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016143103s
I20260807 15:55:15.858289 10654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42236
I20260807 15:55:15.880832 5300 test_util.cc:268] Using random seed: 131264360
I20260807 15:55:15.904877 10654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42264:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:55:15.952034 11074 tablet_service.cc:1467] Processing CreateTablet for tablet 8bdc79826ac34402a88bf64ff671b2cf (DEFAULT_TABLE table=test-workload [id=f8d4bec66cae464ea50ec1b9789a4f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:15.953526 11074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bdc79826ac34402a88bf64ff671b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:15.959838 10769 tablet_service.cc:1467] Processing CreateTablet for tablet 8bdc79826ac34402a88bf64ff671b2cf (DEFAULT_TABLE table=test-workload [id=f8d4bec66cae464ea50ec1b9789a4f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:15.960506 10973 tablet_service.cc:1467] Processing CreateTablet for tablet 8bdc79826ac34402a88bf64ff671b2cf (DEFAULT_TABLE table=test-workload [id=f8d4bec66cae464ea50ec1b9789a4f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:55:15.961313 10769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bdc79826ac34402a88bf64ff671b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:15.961869 10973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bdc79826ac34402a88bf64ff671b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:15.969827 11146 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: Bootstrap starting.
I20260807 15:55:15.977783 11146 tablet_bootstrap.cc:654] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:15.979980 11147 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Bootstrap starting.
I20260807 15:55:15.981689 11149 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: Bootstrap starting.
I20260807 15:55:15.985316 11146 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: No bootstrap required, opened a new log
I20260807 15:55:15.985828 11146 ts_tablet_manager.cc:1397] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260807 15:55:15.988181 11149 tablet_bootstrap.cc:654] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:15.988590 11147 tablet_bootstrap.cc:654] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Neither blocks nor log segments found. Creating new log.
I20260807 15:55:15.988765 11146 raft_consensus.cc:348] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:15.989619 11146 raft_consensus.cc:374] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:15.989940 11146 raft_consensus.cc:729] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68ce8ee81bd4ff0b2d6ed81e69ea9a5, State: Initialized, Role: FOLLOWER
I20260807 15:55:15.990713 11146 consensus_queue.cc:260] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:15.993639 11149 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: No bootstrap required, opened a new log
I20260807 15:55:15.993633 11118 heartbeater.cc:503] Master 127.5.45.62:39233 was elected leader, sending a full tablet report...
I20260807 15:55:15.994120 11146 ts_tablet_manager.cc:1428] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260807 15:55:15.994359 11149 ts_tablet_manager.cc:1397] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260807 15:55:15.995635 11147 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: No bootstrap required, opened a new log
I20260807 15:55:15.996119 11147 ts_tablet_manager.cc:1397] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.012s
I20260807 15:55:15.997301 11149 raft_consensus.cc:348] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:15.998118 11149 raft_consensus.cc:374] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:15.998421 11149 raft_consensus.cc:729] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f6d897fceb4200bb49b3bd0dde5620, State: Initialized, Role: FOLLOWER
I20260807 15:55:15.998673 11147 raft_consensus.cc:348] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:15.999465 11147 raft_consensus.cc:374] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:55:15.999272 11149 consensus_queue.cc:260] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:15.999776 11147 raft_consensus.cc:729] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f1825e113848c5975ccfc9561809b5, State: Initialized, Role: FOLLOWER
I20260807 15:55:16.000816 11147 consensus_queue.cc:260] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:16.002192 11024 heartbeater.cc:503] Master 127.5.45.62:39233 was elected leader, sending a full tablet report...
I20260807 15:55:16.002628 11149 ts_tablet_manager.cc:1428] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260807 15:55:16.005785 10816 heartbeater.cc:503] Master 127.5.45.62:39233 was elected leader, sending a full tablet report...
I20260807 15:55:16.006372 11156 raft_consensus.cc:482] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:55:16.006873 11156 raft_consensus.cc:504] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:16.009294 11147 ts_tablet_manager.cc:1428] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 15:55:16.010429 11156 leader_election.cc:288] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677), 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639)
I20260807 15:55:16.024152 11087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "70f1825e113848c5975ccfc9561809b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" is_pre_election: true
I20260807 15:55:16.024968 11087 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70f1825e113848c5975ccfc9561809b5 in term 0.
I20260807 15:55:16.026108 10740 leader_election.cc:302] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70f1825e113848c5975ccfc9561809b5, e68ce8ee81bd4ff0b2d6ed81e69ea9a5; no voters:
I20260807 15:55:16.026882 11156 raft_consensus.cc:2781] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:55:16.027184 11156 raft_consensus.cc:482] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:55:16.027489 11156 raft_consensus.cc:3037] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:16.030129 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "70f1825e113848c5975ccfc9561809b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f6d897fceb4200bb49b3bd0dde5620" is_pre_election: true
I20260807 15:55:16.030901 10987 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70f1825e113848c5975ccfc9561809b5 in term 0.
I20260807 15:55:16.033124 11156 raft_consensus.cc:504] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:16.034682 11156 leader_election.cc:288] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [CANDIDATE]: Term 1 election: Requested vote from peers e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677), 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639)
I20260807 15:55:16.035549 11087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "70f1825e113848c5975ccfc9561809b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5"
I20260807 15:55:16.035831 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "70f1825e113848c5975ccfc9561809b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f6d897fceb4200bb49b3bd0dde5620"
I20260807 15:55:16.036163 11087 raft_consensus.cc:3037] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:16.036446 10987 raft_consensus.cc:3037] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:55:16.041203 11087 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70f1825e113848c5975ccfc9561809b5 in term 1.
I20260807 15:55:16.041850 10987 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70f1825e113848c5975ccfc9561809b5 in term 1.
I20260807 15:55:16.042335 10740 leader_election.cc:302] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70f1825e113848c5975ccfc9561809b5, e68ce8ee81bd4ff0b2d6ed81e69ea9a5; no voters:
I20260807 15:55:16.043157 11156 raft_consensus.cc:2781] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:55:16.044247 11156 raft_consensus.cc:686] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 1 LEADER]: Becoming Leader. State: Replica: 70f1825e113848c5975ccfc9561809b5, State: Running, Role: LEADER
I20260807 15:55:16.045076 11156 consensus_queue.cc:237] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:16.051750 10653 catalog_manager.cc:5495] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70f1825e113848c5975ccfc9561809b5 (127.5.45.1). New cstate: current_term: 1 leader_uuid: "70f1825e113848c5975ccfc9561809b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } health_report { overall_health: UNKNOWN } } }
I20260807 15:55:16.328367 10769 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:16.330401 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:16.334107 10973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:16.340864 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:16.518446 11156 consensus_queue.cc:1035] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 15:55:16.544939 11158 consensus_queue.cc:1035] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 1 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 0 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 8bdc79826ac34402a88bf64ff671b2cf of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677): RUNNING
70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): RUNNING [LEADER]
87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639): RUNNING
All reported replicas are:
A = e68ce8ee81bd4ff0b2d6ed81e69ea9a5
B = 70f1825e113848c5975ccfc9561809b5
C = 87f6d897fceb4200bb49b3bd0dde5620
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B* C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:16.630579 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:16.661128 10916 heartbeater.cc:503] Master 127.5.45.62:39233 was elected leader, sending a full tablet report...
I20260807 15:55:17.009840 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:17.020918 10769 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:17.030421 10973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:17.031597 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 1 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 0 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 15:55:17.254846 5300 log_verifier.cc:126] Checking tablet 8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:17.318339 5300 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:17.320360 5300 tablet_replica.cc:330] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: stopping tablet replica
I20260807 15:55:17.320978 5300 raft_consensus.cc:2229] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:17.321914 5300 raft_consensus.cc:2258] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:17.549916 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:17.561213 10973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:17.567746 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:17.568603 10769 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 0 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 0 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 8bdc79826ac34402a88bf64ff671b2cf of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677): RUNNING
70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639): RUNNING
All reported replicas are:
A = e68ce8ee81bd4ff0b2d6ed81e69ea9a5
B = 70f1825e113848c5975ccfc9561809b5
C = 87f6d897fceb4200bb49b3bd0dde5620
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:17.801358 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:18.189529 10769 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:18.214470 10973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:18.218441 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:18.220161 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 0 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 0 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 8bdc79826ac34402a88bf64ff671b2cf of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677): RUNNING
70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639): RUNNING
All reported replicas are:
A = e68ce8ee81bd4ff0b2d6ed81e69ea9a5
B = 70f1825e113848c5975ccfc9561809b5
C = 87f6d897fceb4200bb49b3bd0dde5620
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | -1 | Yes
B | [config not available] | | |
C | A B* C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:18.431079 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:18.658425 11320 raft_consensus.cc:482] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70f1825e113848c5975ccfc9561809b5)
I20260807 15:55:18.659081 11320 raft_consensus.cc:504] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:18.666350 11320 leader_election.cc:288] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677), 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043)
I20260807 15:55:18.680713 11330 raft_consensus.cc:482] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70f1825e113848c5975ccfc9561809b5)
I20260807 15:55:18.683187 11330 raft_consensus.cc:504] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:18.692011 11087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "87f6d897fceb4200bb49b3bd0dde5620" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" is_pre_election: true
I20260807 15:55:18.692813 11087 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f6d897fceb4200bb49b3bd0dde5620 in term 1.
I20260807 15:55:18.694562 11330 leader_election.cc:288] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043), 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639)
I20260807 15:55:18.695125 10937 leader_election.cc:302] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87f6d897fceb4200bb49b3bd0dde5620, e68ce8ee81bd4ff0b2d6ed81e69ea9a5; no voters:
I20260807 15:55:18.696372 11320 raft_consensus.cc:2781] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:55:18.696763 11320 raft_consensus.cc:482] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 70f1825e113848c5975ccfc9561809b5)
I20260807 15:55:18.697176 11320 raft_consensus.cc:3037] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:55:18.705739 11320 raft_consensus.cc:504] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:18.708874 11087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "87f6d897fceb4200bb49b3bd0dde5620" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5"
I20260807 15:55:18.709375 11320 leader_election.cc:288] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 election: Requested vote from peers e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677), 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043)
I20260807 15:55:18.709553 11087 raft_consensus.cc:3037] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:55:18.712168 10786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "87f6d897fceb4200bb49b3bd0dde5620" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "70f1825e113848c5975ccfc9561809b5" is_pre_election: true
I20260807 15:55:18.714085 10785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "87f6d897fceb4200bb49b3bd0dde5620" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "70f1825e113848c5975ccfc9561809b5"
W20260807 15:55:18.716042 10931 leader_election.cc:341] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): Illegal state: cannot vote while shut down
W20260807 15:55:18.717039 10931 leader_election.cc:341] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): Illegal state: cannot vote while shut down
I20260807 15:55:18.719383 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87f6d897fceb4200bb49b3bd0dde5620" is_pre_election: true
I20260807 15:55:18.719851 11087 raft_consensus.cc:2454] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f6d897fceb4200bb49b3bd0dde5620 in term 2.
I20260807 15:55:18.720309 10987 raft_consensus.cc:2379] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e68ce8ee81bd4ff0b2d6ed81e69ea9a5 in current term 2: Already voted for candidate 87f6d897fceb4200bb49b3bd0dde5620 in this term.
I20260807 15:55:18.721242 10937 leader_election.cc:302] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 87f6d897fceb4200bb49b3bd0dde5620, e68ce8ee81bd4ff0b2d6ed81e69ea9a5; no voters: 70f1825e113848c5975ccfc9561809b5
I20260807 15:55:18.722635 10785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bdc79826ac34402a88bf64ff671b2cf" candidate_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "70f1825e113848c5975ccfc9561809b5" is_pre_election: true
I20260807 15:55:18.722738 11320 raft_consensus.cc:2781] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 15:55:18.723861 11038 leader_election.cc:341] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): Illegal state: cannot vote while shut down
I20260807 15:55:18.724152 11320 raft_consensus.cc:686] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 LEADER]: Becoming Leader. State: Replica: 87f6d897fceb4200bb49b3bd0dde5620, State: Running, Role: LEADER
I20260807 15:55:18.724380 11038 leader_election.cc:302] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e68ce8ee81bd4ff0b2d6ed81e69ea9a5; no voters: 70f1825e113848c5975ccfc9561809b5, 87f6d897fceb4200bb49b3bd0dde5620
I20260807 15:55:18.725262 11330 raft_consensus.cc:2726] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:55:18.725085 11320 consensus_queue.cc:237] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:18.732992 10653 catalog_manager.cc:5495] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 reported cstate change: term changed from 1 to 2, leader changed from 70f1825e113848c5975ccfc9561809b5 (127.5.45.1) to 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3). New cstate: current_term: 2 leader_uuid: "87f6d897fceb4200bb49b3bd0dde5620" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70f1825e113848c5975ccfc9561809b5" member_type: VOTER last_known_addr { host: "127.5.45.1" port: 38043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } health_report { overall_health: HEALTHY } } }
I20260807 15:55:18.934495 10769 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:18.939805 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:18.942365 10973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:18.947713 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:19.185676 10931 consensus_queue.cc:579] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: The tablet replica hosted on peer 70f1825e113848c5975ccfc9561809b5 has failed
W20260807 15:55:19.186236 10931 consensus_peers.cc:487] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 -> Peer 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): Couldn't send request to peer 70f1825e113848c5975ccfc9561809b5. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 0 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 1 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 8bdc79826ac34402a88bf64ff671b2cf of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 (127.5.45.4:41677): RUNNING
70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639): RUNNING [LEADER]
All reported replicas are:
A = e68ce8ee81bd4ff0b2d6ed81e69ea9a5
B = 70f1825e113848c5975ccfc9561809b5
C = 87f6d897fceb4200bb49b3bd0dde5620
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 2 | -1 | Yes
B | [config not available] | | |
C | A B C* | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260807 15:55:19.210500 5300 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260807 15:55:19.275547 11087 raft_consensus.cc:1261] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Refusing update from remote peer 87f6d897fceb4200bb49b3bd0dde5620: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 15:55:19.276983 11340 consensus_queue.cc:1035] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 15:55:19.293869 10987 consensus_queue.cc:237] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } }
I20260807 15:55:19.297315 11087 raft_consensus.cc:1261] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Refusing update from remote peer 87f6d897fceb4200bb49b3bd0dde5620: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 15:55:19.298594 11340 consensus_queue.cc:1035] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 15:55:19.303643 11320 raft_consensus.cc:2932] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 70f1825e113848c5975ccfc9561809b5 (127.5.45.1) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } }
I20260807 15:55:19.304983 11087 raft_consensus.cc:2932] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 70f1825e113848c5975ccfc9561809b5 (127.5.45.1) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } }
I20260807 15:55:19.310465 10639 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8bdc79826ac34402a88bf64ff671b2cf with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 15:55:19.314177 10653 catalog_manager.cc:5495] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 reported cstate change: config changed from index -1 to 3, VOTER 70f1825e113848c5975ccfc9561809b5 (127.5.45.1) evicted. New cstate: current_term: 2 leader_uuid: "87f6d897fceb4200bb49b3bd0dde5620" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } health_report { overall_health: HEALTHY } } }
I20260807 15:55:19.325732 10769 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bdc79826ac34402a88bf64ff671b2cf with delete_type TABLET_DATA_TOMBSTONED (TS 70f1825e113848c5975ccfc9561809b5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:55210
I20260807 15:55:19.327944 11382 ts_tablet_manager.cc:1905] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 15:55:19.329078 10987 consensus_queue.cc:237] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } }
I20260807 15:55:19.336308 11087 raft_consensus.cc:1261] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Refusing update from remote peer 87f6d897fceb4200bb49b3bd0dde5620: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 15:55:19.337978 11377 consensus_queue.cc:1035] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 15:55:19.344918 11382 ts_tablet_manager.cc:1918] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260807 15:55:19.344211 11320 raft_consensus.cc:2932] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 1c21870d2b0946a4b0718ce8d4f49106 (127.5.45.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } } }
I20260807 15:55:19.345561 11382 log.cc:1196] T 8bdc79826ac34402a88bf64ff671b2cf P 70f1825e113848c5975ccfc9561809b5: Deleting WAL directory at /tmp/dist-test-taskytNvUu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786118049188364-5300-0/minicluster-data/ts-0-root/wals/8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:19.347450 10636 catalog_manager.cc:4841] TS 70f1825e113848c5975ccfc9561809b5 (127.5.45.1:38043): tablet 8bdc79826ac34402a88bf64ff671b2cf (table test-workload [id=f8d4bec66cae464ea50ec1b9789a4f28]) successfully deleted
I20260807 15:55:19.347592 11087 raft_consensus.cc:2932] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 1c21870d2b0946a4b0718ce8d4f49106 (127.5.45.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } } }
W20260807 15:55:19.350226 10934 consensus_peers.cc:487] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 -> Peer 1c21870d2b0946a4b0718ce8d4f49106 (127.5.45.2:36669): Couldn't send request to peer 1c21870d2b0946a4b0718ce8d4f49106. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bdc79826ac34402a88bf64ff671b2cf. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:55:19.353390 10639 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8bdc79826ac34402a88bf64ff671b2cf with cas_config_opid_index 3: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260807 15:55:19.357280 10653 catalog_manager.cc:5495] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 reported cstate change: config changed from index 3 to 4, NON_VOTER 1c21870d2b0946a4b0718ce8d4f49106 (127.5.45.2) added. New cstate: current_term: 2 leader_uuid: "87f6d897fceb4200bb49b3bd0dde5620" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 15:55:19.929805 11410 ts_tablet_manager.cc:927] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Initiating tablet copy from peer 87f6d897fceb4200bb49b3bd0dde5620 (127.5.45.3:35639)
I20260807 15:55:19.931408 11410 tablet_copy_client.cc:316] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: tablet copy: Beginning tablet copy session from remote peer at address 127.5.45.3:35639
I20260807 15:55:19.945921 11002 tablet_copy_service.cc:140] P 87f6d897fceb4200bb49b3bd0dde5620: Received BeginTabletCopySession request for tablet 8bdc79826ac34402a88bf64ff671b2cf from peer 1c21870d2b0946a4b0718ce8d4f49106 ({username='slave'} at 127.0.0.1:41484)
I20260807 15:55:19.946429 11002 tablet_copy_service.cc:161] P 87f6d897fceb4200bb49b3bd0dde5620: Beginning new tablet copy session on tablet 8bdc79826ac34402a88bf64ff671b2cf from peer 1c21870d2b0946a4b0718ce8d4f49106 at {username='slave'} at 127.0.0.1:41484: session id = 1c21870d2b0946a4b0718ce8d4f49106-8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:19.958201 11002 tablet_copy_source_session.cc:205] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:55:19.962208 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bdc79826ac34402a88bf64ff671b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:55:19.979382 10864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:19.997128 10973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260807 15:55:20.010380 11410 tablet_copy_client.cc:791] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: tablet copy: Starting download of 0 data blocks...
I20260807 15:55:20.011205 11410 tablet_copy_client.cc:655] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: tablet copy: Starting download of 1 WAL segments...
I20260807 15:55:20.015838 11410 tablet_copy_client.cc:531] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:55:20.025103 11410 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Bootstrap starting.
I20260807 15:55:20.030268 10769 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:20.044467 11074 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260807 15:55:20.094448 11410 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:55:20.100561 11410 tablet_bootstrap.cc:492] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Bootstrap complete.
I20260807 15:55:20.101732 11410 ts_tablet_manager.cc:1397] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Time spent bootstrapping tablet: real 0.077s user 0.025s sys 0.029s
I20260807 15:55:20.104961 11410 raft_consensus.cc:348] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } }
I20260807 15:55:20.117528 11410 raft_consensus.cc:729] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1c21870d2b0946a4b0718ce8d4f49106, State: Initialized, Role: LEARNER
I20260807 15:55:20.118366 11410 consensus_queue.cc:260] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e68ce8ee81bd4ff0b2d6ed81e69ea9a5" member_type: VOTER last_known_addr { host: "127.5.45.4" port: 41677 } } peers { permanent_uuid: "87f6d897fceb4200bb49b3bd0dde5620" member_type: VOTER last_known_addr { host: "127.5.45.3" port: 35639 } } peers { permanent_uuid: "1c21870d2b0946a4b0718ce8d4f49106" member_type: NON_VOTER last_known_addr { host: "127.5.45.2" port: 36669 } attrs { promote: true } }
I20260807 15:55:20.143101 11410 ts_tablet_manager.cc:1428] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: Time spent starting tablet: real 0.041s user 0.010s sys 0.002s
I20260807 15:55:20.151727 11002 tablet_copy_service.cc:342] P 87f6d897fceb4200bb49b3bd0dde5620: Request end of tablet copy session 1c21870d2b0946a4b0718ce8d4f49106-8bdc79826ac34402a88bf64ff671b2cf received from {username='slave'} at 127.0.0.1:41484
I20260807 15:55:20.152211 11002 tablet_copy_service.cc:433] P 87f6d897fceb4200bb49b3bd0dde5620: ending tablet copy session 1c21870d2b0946a4b0718ce8d4f49106-8bdc79826ac34402a88bf64ff671b2cf on tablet 8bdc79826ac34402a88bf64ff671b2cf with peer 1c21870d2b0946a4b0718ce8d4f49106
Master Summary
UUID | Address | Status
----------------------------------+-------------------+---------
59d7c6067c16480c9c4a4855ad7c728f | 127.5.45.62:39233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
1c21870d2b0946a4b0718ce8d4f49106 | 127.5.45.2:36669 | HEALTHY | <none> | 0 | 0
70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | HEALTHY | <none> | 0 | 0
87f6d897fceb4200bb49b3bd0dde5620 | 127.5.45.3:35639 | HEALTHY | <none> | 1 | 0
e68ce8ee81bd4ff0b2d6ed81e69ea9a5 | 127.5.45.4:41677 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+------------------+---------------
Small | 70f1825e113848c5975ccfc9561809b5 | 127.5.45.1:38043 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 15:55:20.279063 5300 log_verifier.cc:126] Checking tablet 8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:20.339145 5300 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 8bdc79826ac34402a88bf64ff671b2cf
I20260807 15:55:20.341687 5300 tablet_server.cc:178] TabletServer@127.5.45.1:0 shutting down...
I20260807 15:55:20.371598 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:20.391078 5300 tablet_server.cc:194] TabletServer@127.5.45.1:0 shutdown complete.
I20260807 15:55:20.401079 5300 tablet_server.cc:178] TabletServer@127.5.45.2:0 shutting down...
I20260807 15:55:20.423635 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:20.424437 5300 tablet_replica.cc:330] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106: stopping tablet replica
I20260807 15:55:20.424991 5300 raft_consensus.cc:2229] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106 [term 2 LEARNER]: Raft consensus shutting down.
I20260807 15:55:20.425469 5300 raft_consensus.cc:2258] T 8bdc79826ac34402a88bf64ff671b2cf P 1c21870d2b0946a4b0718ce8d4f49106 [term 2 LEARNER]: Raft consensus is shut down!
I20260807 15:55:20.446139 5300 tablet_server.cc:194] TabletServer@127.5.45.2:0 shutdown complete.
I20260807 15:55:20.456626 5300 tablet_server.cc:178] TabletServer@127.5.45.3:0 shutting down...
I20260807 15:55:20.479526 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:20.480192 5300 tablet_replica.cc:330] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620: stopping tablet replica
I20260807 15:55:20.480783 5300 raft_consensus.cc:2229] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 LEADER]: Raft consensus shutting down.
I20260807 15:55:20.481815 5300 raft_consensus.cc:2258] T 8bdc79826ac34402a88bf64ff671b2cf P 87f6d897fceb4200bb49b3bd0dde5620 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:20.502146 5300 tablet_server.cc:194] TabletServer@127.5.45.3:0 shutdown complete.
I20260807 15:55:20.512208 5300 tablet_server.cc:178] TabletServer@127.5.45.4:0 shutting down...
I20260807 15:55:20.533675 5300 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260807 15:55:20.534386 5300 tablet_replica.cc:330] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5: stopping tablet replica
I20260807 15:55:20.535109 5300 raft_consensus.cc:2229] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 15:55:20.535676 5300 raft_consensus.cc:2258] T 8bdc79826ac34402a88bf64ff671b2cf P e68ce8ee81bd4ff0b2d6ed81e69ea9a5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260807 15:55:20.866854 11113 debug-util.cc:398] Leaking SignalData structure 0x7b080024db40 after lost signal to thread 5304
W20260807 15:55:21.554234 5300 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 11113)
W20260807 15:55:22.554858 5300 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 11113)
W20260807 15:55:23.555408 5300 thread.cc:536] Waited for 3000ms trying to join with diag-logger (tid 11113)
I20260807 15:55:23.997394 5300 tablet_server.cc:194] TabletServer@127.5.45.4:0 shutdown complete.
I20260807 15:55:24.014935 5300 master.cc:537] Master@127.5.45.62:39233 shutting down...
I20260807 15:55:24.041882 5300 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:55:24.042723 5300 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:55:24.043044 5300 tablet_replica.cc:330] T 00000000000000000000000000000000 P 59d7c6067c16480c9c4a4855ad7c728f: stopping tablet replica
I20260807 15:55:24.063838 5300 master.cc:552] Master@127.5.45.62:39233 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (8869 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (14005 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (74750 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260807 15:55:24.113955 5300 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~14 seconds ago
I20260807 15:55:24.114346 5300 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 15 messages since previous log ~19 seconds ago
I20260807 15:55:24.114567 5300 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 100 messages since previous log ~65 seconds ago
I20260807 15:55:24.114851 5300 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 8 messages since previous log ~18 seconds ago