[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260811 01:57:06.458556 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:43307
I20260811 01:57:06.459659 10906 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:57:06.460417 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:06.464444 10921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:06.464406 10916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:06.464538 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:06.464584 10919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:06.465068 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:06.465200 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:06.465255 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413426465251 us; error 0 us; skew 500 ppm
I20260811 01:57:06.466871 10906 webserver.cc:533] Webserver started at http://127.10.166.190:43421/ using document root <none> and password file <none>
I20260811 01:57:06.467314 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:06.467370 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:06.467569 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:06.469201 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "843d2c61fc2342cab12a05d74c97a092"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.472415 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:57:06.474200 10927 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.475163 10906 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 01:57:06.475257 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "843d2c61fc2342cab12a05d74c97a092"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.475353 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:06.486157 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:06.486632 10906 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:57:06.486780 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:06.493569 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:43307
I20260811 01:57:06.493582 11003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:43307 every 8 connection(s)
I20260811 01:57:06.495482 11004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:06.500353 11004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: Bootstrap starting.
I20260811 01:57:06.502805 11004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:06.503574 11004 log.cc:826] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: Log is configured to *not* fsync() on all Append() calls
I20260811 01:57:06.505070 11004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: No bootstrap required, opened a new log
I20260811 01:57:06.507738 11004 raft_consensus.cc:359] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER }
I20260811 01:57:06.507889 11004 raft_consensus.cc:385] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:06.507963 11004 raft_consensus.cc:740] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843d2c61fc2342cab12a05d74c97a092, State: Initialized, Role: FOLLOWER
I20260811 01:57:06.508453 11004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER }
I20260811 01:57:06.508574 11004 raft_consensus.cc:399] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:06.508633 11004 raft_consensus.cc:493] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:06.508718 11004 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:06.509384 11004 raft_consensus.cc:515] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER }
I20260811 01:57:06.509666 11004 leader_election.cc:304] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843d2c61fc2342cab12a05d74c97a092; no voters:
I20260811 01:57:06.509895 11004 leader_election.cc:290] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:06.510012 11009 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:06.510226 11009 raft_consensus.cc:697] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 1 LEADER]: Becoming Leader. State: Replica: 843d2c61fc2342cab12a05d74c97a092, State: Running, Role: LEADER
I20260811 01:57:06.510474 11009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER }
I20260811 01:57:06.510772 11004 sys_catalog.cc:565] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:06.511924 11012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843d2c61fc2342cab12a05d74c97a092" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER } }
I20260811 01:57:06.512038 11012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:06.512326 11013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843d2c61fc2342cab12a05d74c97a092. Latest consensus state: current_term: 1 leader_uuid: "843d2c61fc2342cab12a05d74c97a092" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843d2c61fc2342cab12a05d74c97a092" member_type: VOTER } }
I20260811 01:57:06.512408 11013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:06.512392 11020 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:06.515022 11020 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:06.515285 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:06.519672 11020 catalog_manager.cc:1383] Generated new cluster ID: 0bab434175104b008afd6c2a3160c33c
I20260811 01:57:06.519789 11020 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:06.542366 11020 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:06.543144 11020 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:06.550153 11020 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: Generated new TSK 0
I20260811 01:57:06.550698 11020 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:06.580137 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:06.582343 11038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:06.582454 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:06.582386 11036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:06.582587 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:06.582741 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:06.582779 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:06.582799 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413426582799 us; error 0 us; skew 500 ppm
I20260811 01:57:06.583639 10906 webserver.cc:533] Webserver started at http://127.10.166.129:40149/ using document root <none> and password file <none>
I20260811 01:57:06.583806 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:06.583855 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:06.583927 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:06.584301 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "067a4448010f4c4096f6a6881e20ad5e"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.585704 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:06.586604 11047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.586835 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:06.586898 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "067a4448010f4c4096f6a6881e20ad5e"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.586961 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:06.592835 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:06.593185 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:06.593555 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:06.594575 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:06.594626 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.594663 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:06.594691 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.600664 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:46465
I20260811 01:57:06.600709 11139 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:46465 every 8 connection(s)
I20260811 01:57:06.601514 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:06.603119 11148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:06.603539 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:06.603598 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:06.603996 11142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:06.604225 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:06.604278 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:06.604296 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413426604296 us; error 0 us; skew 500 ppm
I20260811 01:57:06.605126 10906 webserver.cc:533] Webserver started at http://127.10.166.130:45799/ using document root <none> and password file <none>
I20260811 01:57:06.605264 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:06.605304 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:06.605358 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:06.605676 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "bd948c647ec746139d386d7d38522a90"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.607300 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:06.608497 11157 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.608749 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:06.608829 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "bd948c647ec746139d386d7d38522a90"
format_stamp: "Formatted at 2026-08-11 01:57:06 on dist-test-slave-1l3l"
I20260811 01:57:06.608896 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:06.617887 11140 heartbeater.cc:344] Connected to a master server at 127.10.166.190:43307
I20260811 01:57:06.618100 11140 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:06.618479 11140 heartbeater.cc:507] Master 127.10.166.190:43307 requested a full tablet report, sending...
I20260811 01:57:06.619684 10951 ts_manager.cc:194] Registered new tserver with Master: 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129:46465)
I20260811 01:57:06.620832 10951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52350
I20260811 01:57:06.629070 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:06.629514 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:06.629848 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:06.630419 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:06.630481 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.630533 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:06.630563 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:06.636945 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:45427
I20260811 01:57:06.636979 11256 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:45427 every 8 connection(s)
I20260811 01:57:06.642061 11257 heartbeater.cc:344] Connected to a master server at 127.10.166.190:43307
I20260811 01:57:06.642154 11257 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:06.642333 11257 heartbeater.cc:507] Master 127.10.166.190:43307 requested a full tablet report, sending...
I20260811 01:57:06.642923 10951 ts_manager.cc:194] Registered new tserver with Master: bd948c647ec746139d386d7d38522a90 (127.10.166.130:45427)
I20260811 01:57:06.643201 10906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.005702304s
I20260811 01:57:06.643517 10951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52356
I20260811 01:57:06.651908 10951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:57:06.655095 10951 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260811 01:57:06.667846 11195 tablet_service.cc:1511] Processing CreateTablet for tablet fd68c34d935e49f69b64fa5f1c20bb2d (DEFAULT_TABLE table=test-table [id=31670facde4d45828dcbde5d5e2085cc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:06.667980 11084 tablet_service.cc:1511] Processing CreateTablet for tablet fd68c34d935e49f69b64fa5f1c20bb2d (DEFAULT_TABLE table=test-table [id=31670facde4d45828dcbde5d5e2085cc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:06.668296 11195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd68c34d935e49f69b64fa5f1c20bb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:06.668301 11084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd68c34d935e49f69b64fa5f1c20bb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:06.668524 11085 tablet_service.cc:1511] Processing CreateTablet for tablet d2ec78a0139d4f6187d9c006064aa70a (DEFAULT_TABLE table=test-table [id=31670facde4d45828dcbde5d5e2085cc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:06.668749 11085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ec78a0139d4f6187d9c006064aa70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:06.667846 11196 tablet_service.cc:1511] Processing CreateTablet for tablet d2ec78a0139d4f6187d9c006064aa70a (DEFAULT_TABLE table=test-table [id=31670facde4d45828dcbde5d5e2085cc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:06.671080 11196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ec78a0139d4f6187d9c006064aa70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:06.671218 11276 tablet_bootstrap.cc:492] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: Bootstrap starting.
I20260811 01:57:06.672037 11277 tablet_bootstrap.cc:492] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: Bootstrap starting.
I20260811 01:57:06.672129 11276 tablet_bootstrap.cc:654] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:06.673038 11277 tablet_bootstrap.cc:654] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:06.673188 11276 tablet_bootstrap.cc:492] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: No bootstrap required, opened a new log
I20260811 01:57:06.673281 11276 ts_tablet_manager.cc:1403] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:06.673696 11276 raft_consensus.cc:359] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:06.673815 11276 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Initialized, Role: FOLLOWER
I20260811 01:57:06.673916 11276 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:06.674154 11277 tablet_bootstrap.cc:492] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: No bootstrap required, opened a new log
I20260811 01:57:06.674223 11277 ts_tablet_manager.cc:1403] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:06.674270 11276 ts_tablet_manager.cc:1434] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:06.674440 11140 heartbeater.cc:499] Master 127.10.166.190:43307 was elected leader, sending a full tablet report...
I20260811 01:57:06.674475 11276 tablet_bootstrap.cc:492] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: Bootstrap starting.
I20260811 01:57:06.675359 11276 tablet_bootstrap.cc:654] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:06.675478 11277 raft_consensus.cc:359] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:06.675561 11277 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Initialized, Role: FOLLOWER
I20260811 01:57:06.675647 11277 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:06.675989 11277 ts_tablet_manager.cc:1434] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:06.676177 11257 heartbeater.cc:499] Master 127.10.166.190:43307 was elected leader, sending a full tablet report...
I20260811 01:57:06.676222 11277 tablet_bootstrap.cc:492] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: Bootstrap starting.
I20260811 01:57:06.676954 11277 tablet_bootstrap.cc:654] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:06.677002 11276 tablet_bootstrap.cc:492] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: No bootstrap required, opened a new log
I20260811 01:57:06.677067 11276 ts_tablet_manager.cc:1403] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 01:57:06.677418 11276 raft_consensus.cc:359] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:06.677515 11276 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Initialized, Role: FOLLOWER
I20260811 01:57:06.677596 11276 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:06.677820 11276 ts_tablet_manager.cc:1434] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:06.678566 11277 tablet_bootstrap.cc:492] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: No bootstrap required, opened a new log
I20260811 01:57:06.678640 11277 ts_tablet_manager.cc:1403] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:06.679015 11277 raft_consensus.cc:359] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:06.679093 11277 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Initialized, Role: FOLLOWER
I20260811 01:57:06.679163 11277 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:06.679364 11277 ts_tablet_manager.cc:1434] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:06.691969 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:06.692020 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Emulating election...
I20260811 01:57:06.692054 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:06.692816 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:06.692963 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:06.693849 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:06.695173 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 1 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:06.700657 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:06.802021 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
W20260811 01:57:06.851389 11141 tablet.cc:2406] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:57:07.003445 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:07.079946 11098 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:07.081508 11283 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:07.304920 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:07.304998 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Emulating election...
I20260811 01:57:07.305037 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:07.305927 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:07.306107 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:07.306689 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:07.307520 10950 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 1 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:07.307905 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:07.409194 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:07.610641 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:07.868340 11098 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:07.869648 11283 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:08.112195 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:08.112274 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Emulating election...
I20260811 01:57:08.112315 10906 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:08.112340 10906 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:08.112465 10906 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:08.112721 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:08.113636 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 2 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:08.113783 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:08.114225 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:08.115028 10950 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:08.115559 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260811 01:57:08.216970 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260811 01:57:08.418515 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 1.1 2.2
I20260811 01:57:08.622762 11098 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:08.623809 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 2 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 01:57:08.624193 11283 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:08.719942 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:08.720022 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 1 FOLLOWER]: Emulating election...
I20260811 01:57:08.720059 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:08.720932 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 2 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:08.721092 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:08.721832 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:08.722698 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 1 to 2, leader changed from bd948c647ec746139d386d7d38522a90 (127.10.166.130) to 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129). New cstate: current_term: 2 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:08.723147 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:08.824532 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:08.937057 11098 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 2 LEADER]: Rejecting Update request from peer bd948c647ec746139d386d7d38522a90 for earlier term 1. Current term is 2. Ops: []
I20260811 01:57:08.937492 11317 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:08.937671 11317 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer 067a4448010f4c4096f6a6881e20ad5e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260811 01:57:08.937738 11283 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:08.937819 11283 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:08.937943 11283 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:08.938077 11283 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:09.026202 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:09.140655 11215 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 2 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 01:57:09.141084 11316 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:09.527621 10906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:09.527720 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 2 FOLLOWER]: Emulating election...
I20260811 01:57:09.527761 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:09.528661 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 3 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:09.528836 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:09.529335 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:09.530175 10950 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 2 to 3, leader changed from bd948c647ec746139d386d7d38522a90 (127.10.166.130) to 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129). New cstate: current_term: 3 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:09.530686 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:09.547983 11098 raft_consensus.cc:1240] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 3 LEADER]: Rejecting Update request from peer bd948c647ec746139d386d7d38522a90 for earlier term 2. Current term is 3. Ops: []
I20260811 01:57:09.548436 11283 consensus_queue.cc:1059] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:09.548660 11283 consensus_queue.cc:1243] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer 067a4448010f4c4096f6a6881e20ad5e log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260811 01:57:09.548731 11345 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:09.548784 11345 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:09.548884 11345 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:09.549026 11345 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:09.632068 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:09.833515 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:10.085951 11215 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 3 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:10.086351 11336 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:10.134864 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:10.134941 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 3 FOLLOWER]: Emulating election...
I20260811 01:57:10.134977 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:10.135907 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:10.136070 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:10.136503 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:10.137383 10951 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 3 to 4, leader changed from 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129) to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 4 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:10.137712 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:10.239055 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:10.440606 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:10.511325 11215 raft_consensus.cc:1240] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Rejecting Update request from peer 067a4448010f4c4096f6a6881e20ad5e for earlier term 3. Current term is 4. Ops: []
I20260811 01:57:10.511742 11350 consensus_queue.cc:1059] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:10.511884 11350 consensus_queue.cc:1243] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer bd948c647ec746139d386d7d38522a90 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260811 01:57:10.511950 11336 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 3 LEADER]: Stepping down as leader of term 3
I20260811 01:57:10.512025 11336 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:10.512136 11336 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:10.512279 11336 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:10.602150 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 4 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260811 01:57:10.602596 11346 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:10.941969 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:10.942044 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Emulating election...
I20260811 01:57:10.942096 10906 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Stepping down as leader of term 4
I20260811 01:57:10.942132 10906 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:10.942256 10906 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:10.942410 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:10.943303 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 5 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:10.943465 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:10.943979 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:10.944859 10951 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 4 to 5. New cstate: current_term: 5 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:10.945183 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:11.046686 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:11.248060 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:11.370865 11098 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:11.371882 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 5 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260811 01:57:11.372354 11346 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:11.549479 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:11.549578 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 2 FOLLOWER]: Emulating election...
I20260811 01:57:11.549633 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:11.550701 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:11.550880 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:11.551445 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:11.552307 10951 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 2 to 3, leader changed from 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129) to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 3 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:11.552670 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 3.3
I20260811 01:57:11.653877 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 3.3
I20260811 01:57:11.845157 11215 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 LEADER]: Rejecting Update request from peer 067a4448010f4c4096f6a6881e20ad5e for earlier term 2. Current term is 3. Ops: []
I20260811 01:57:11.845588 11366 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:11.845880 11366 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer bd948c647ec746139d386d7d38522a90 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260811 01:57:11.846010 11377 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:11.846079 11377 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:11.846192 11377 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:11.846335 11377 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:11.855067 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 3.3
I20260811 01:57:12.044692 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 3 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:12.045110 11346 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:12.356383 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:12.356462 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 LEADER]: Emulating election...
I20260811 01:57:12.356500 10906 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 01:57:12.356530 10906 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:12.356642 10906 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:12.356781 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:12.357698 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:12.357846 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:12.358336 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:12.359169 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:12.359658 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:12.461050 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:12.662413 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:12.818800 11098 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:12.819988 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 4 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260811 01:57:12.820398 11381 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:12.964010 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:12.964097 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 4 FOLLOWER]: Emulating election...
I20260811 01:57:12.964145 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:12.965040 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:12.965204 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:12.965669 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:12.966714 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 4 to 5, leader changed from bd948c647ec746139d386d7d38522a90 (127.10.166.130) to 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129). New cstate: current_term: 5 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:12.966970 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 4.4
I20260811 01:57:13.068584 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 4.4
I20260811 01:57:13.270212 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 4.4
I20260811 01:57:13.305996 11098 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 LEADER]: Rejecting Update request from peer bd948c647ec746139d386d7d38522a90 for earlier term 4. Current term is 5. Ops: []
I20260811 01:57:13.306425 11381 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:13.306591 11381 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer 067a4448010f4c4096f6a6881e20ad5e log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260811 01:57:13.306654 11392 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Stepping down as leader of term 4
I20260811 01:57:13.306708 11392 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:13.306803 11392 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:13.306954 11392 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:13.469409 11215 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 5 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260811 01:57:13.469851 11404 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:13.771741 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:13.771832 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 LEADER]: Emulating election...
I20260811 01:57:13.771873 10906 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 LEADER]: Stepping down as leader of term 5
I20260811 01:57:13.771893 10906 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:13.771999 10906 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:13.772143 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:13.773070 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:13.773241 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:13.773697 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:13.774580 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 5 to 6. New cstate: current_term: 6 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:13.775000 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:13.876477 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:14.077826 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:14.367130 11215 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:14.368201 11215 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260811 01:57:14.368606 11413 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:14.379077 10906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:14.379132 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 5 FOLLOWER]: Emulating election...
I20260811 01:57:14.379173 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:14.380015 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:14.380146 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:14.380587 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:14.381361 10948 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 5 to 6, leader changed from bd948c647ec746139d386d7d38522a90 (127.10.166.130) to 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129). New cstate: current_term: 6 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:14.381752 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:14.483150 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:14.612190 11098 raft_consensus.cc:1240] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Rejecting Update request from peer bd948c647ec746139d386d7d38522a90 for earlier term 5. Current term is 6. Ops: []
I20260811 01:57:14.612653 11392 consensus_queue.cc:1059] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:14.612936 11392 consensus_queue.cc:1243] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer 067a4448010f4c4096f6a6881e20ad5e log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260811 01:57:14.613046 11422 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 5 LEADER]: Stepping down as leader of term 5
I20260811 01:57:14.613121 11422 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:14.613240 11422 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:14.613379 11422 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:14.684633 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 5.5
I20260811 01:57:14.906534 11215 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260811 01:57:14.906991 11423 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:15.186097 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:15.186182 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Emulating election...
I20260811 01:57:15.186221 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:15.187134 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:15.187315 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:15.187865 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:15.188892 10950 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 6 to 7, leader changed from 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129) to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 7 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:15.189090 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:15.290470 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:15.466858 11215 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Rejecting Update request from peer 067a4448010f4c4096f6a6881e20ad5e for earlier term 6. Current term is 7. Ops: []
I20260811 01:57:15.467272 11423 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:15.467495 11423 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer bd948c647ec746139d386d7d38522a90 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260811 01:57:15.467559 11423 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Stepping down as leader of term 6
I20260811 01:57:15.467603 11423 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:15.467729 11423 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:15.467866 11423 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:15.492162 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:15.581982 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 7 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260811 01:57:15.582392 11422 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:15.793520 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:15.793599 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Emulating election...
I20260811 01:57:15.793638 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:15.794790 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:15.794962 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:15.795413 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:15.795462 11215 raft_consensus.cc:1240] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Rejecting Update request from peer 067a4448010f4c4096f6a6881e20ad5e for earlier term 6. Current term is 7. Ops: []
I20260811 01:57:15.796145 11423 consensus_queue.cc:1059] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:15.796293 11423 consensus_queue.cc:1243] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer bd948c647ec746139d386d7d38522a90 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260811 01:57:15.796363 11423 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Stepping down as leader of term 6
I20260811 01:57:15.796406 11423 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:15.796502 11423 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:15.796576 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:15.796562 10948 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 6 to 7, leader changed from 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129) to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 7 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:15.796622 11423 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:15.897979 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:16.099308 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 6.6 7.7
I20260811 01:57:16.212247 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 7 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260811 01:57:16.212687 11441 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:16.600914 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:16.601009 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Emulating election...
I20260811 01:57:16.601053 10906 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Stepping down as leader of term 7
I20260811 01:57:16.601075 10906 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:16.601182 10906 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:16.601336 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:16.602384 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:16.602548 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:16.603152 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:16.604177 10948 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 7 to 8. New cstate: current_term: 8 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:16.604573 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:16.706001 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:16.907410 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:17.091518 11098 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:17.092504 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 8 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 01:57:17.092900 11441 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:57:17.208767 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:17.208848 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Emulating election...
I20260811 01:57:17.208884 10906 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Stepping down as leader of term 7
I20260811 01:57:17.208909 10906 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:17.209015 10906 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:17.209151 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:17.210036 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:17.210196 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:17.210750 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:17.211602 10948 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 7 to 8. New cstate: current_term: 8 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:17.211985 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:17.313252 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:17.514686 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 8.8
I20260811 01:57:17.703442 11098 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:17.704594 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 8 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 01:57:17.705008 11464 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:57:18.016196 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:18.016278 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Emulating election...
I20260811 01:57:18.016317 10906 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Stepping down as leader of term 8
I20260811 01:57:18.016336 10906 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:18.016440 10906 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:18.016603 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:18.017513 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 9 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:18.017658 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:18.018139 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:18.018880 10950 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 8 to 9. New cstate: current_term: 9 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:18.019479 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:18.120730 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:18.322090 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:18.545712 11098 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:18.546656 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260811 01:57:18.547071 11464 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 01:57:18.623545 10906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20260811 01:57:18.623625 10906 raft_consensus.cc:423] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Emulating election...
I20260811 01:57:18.623662 10906 raft_consensus.cc:3055] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Stepping down as leader of term 8
I20260811 01:57:18.623685 10906 raft_consensus.cc:740] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:18.623805 10906 consensus_queue.cc:260] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:18.623951 10906 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:18.624814 10906 raft_consensus.cc:697] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 9 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:18.624963 10906 consensus_queue.cc:237] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } }
I20260811 01:57:18.625511 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:18.626365 10950 catalog_manager.cc:5719] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 8 to 9. New cstate: current_term: 9 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:18.626978 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:18.728336 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:18.929817 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 9.9
I20260811 01:57:19.161972 11098 raft_consensus.cc:3060] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:19.162988 11098 raft_consensus.cc:1275] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260811 01:57:19.163499 11495 consensus_queue.cc:1048] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 01:57:19.431380 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:19.431473 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Emulating election...
I20260811 01:57:19.431529 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:19.432554 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 10 LEADER]: Becoming Leader. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:19.432732 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:19.433214 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:19.434123 10950 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e reported cstate change: term changed from 9 to 10, leader changed from bd948c647ec746139d386d7d38522a90 (127.10.166.130) to 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129). New cstate: current_term: 10 leader_uuid: "067a4448010f4c4096f6a6881e20ad5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:19.434615 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 9.9
I20260811 01:57:19.535981 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 9.9
I20260811 01:57:19.687798 11098 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 10 LEADER]: Rejecting Update request from peer bd948c647ec746139d386d7d38522a90 for earlier term 9. Current term is 10. Ops: []
I20260811 01:57:19.688298 11498 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 01:57:19.688552 11498 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Peer 067a4448010f4c4096f6a6881e20ad5e log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260811 01:57:19.688642 11498 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 9 LEADER]: Stepping down as leader of term 9
I20260811 01:57:19.688683 11498 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:19.688800 11498 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:19.688938 11498 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:19.737349 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 9.9
I20260811 01:57:19.879165 11215 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 10 FOLLOWER]: Refusing update from remote peer 067a4448010f4c4096f6a6881e20ad5e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260811 01:57:19.879617 11496 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 01:57:20.038671 10906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20260811 01:57:20.038753 10906 raft_consensus.cc:423] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 10 FOLLOWER]: Emulating election...
I20260811 01:57:20.038789 10906 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 10 FOLLOWER]: Advancing to term 11
I20260811 01:57:20.039750 10906 raft_consensus.cc:697] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 11 LEADER]: Becoming Leader. State: Replica: bd948c647ec746139d386d7d38522a90, State: Running, Role: LEADER
I20260811 01:57:20.039929 10906 consensus_queue.cc:237] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:20.040382 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:20.041208 10950 catalog_manager.cc:5719] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 reported cstate change: term changed from 10 to 11, leader changed from 067a4448010f4c4096f6a6881e20ad5e (127.10.166.129) to bd948c647ec746139d386d7d38522a90 (127.10.166.130). New cstate: current_term: 11 leader_uuid: "bd948c647ec746139d386d7d38522a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:20.041532 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 11.11
I20260811 01:57:20.142992 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 11.11
I20260811 01:57:20.259406 11215 raft_consensus.cc:1240] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 11 LEADER]: Rejecting Update request from peer 067a4448010f4c4096f6a6881e20ad5e for earlier term 10. Current term is 11. Ops: []
I20260811 01:57:20.259791 11510 consensus_queue.cc:1059] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260811 01:57:20.259975 11510 consensus_queue.cc:1243] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [LEADER]: Peer bd948c647ec746139d386d7d38522a90 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260811 01:57:20.260042 11496 raft_consensus.cc:3055] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 10 LEADER]: Stepping down as leader of term 10
I20260811 01:57:20.260098 11496 raft_consensus.cc:740] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 067a4448010f4c4096f6a6881e20ad5e, State: Running, Role: LEADER
I20260811 01:57:20.260195 11496 consensus_queue.cc:260] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd948c647ec746139d386d7d38522a90" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 45427 } } peers { permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 } }
I20260811 01:57:20.260306 11496 raft_consensus.cc:3060] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 10 FOLLOWER]: Advancing to term 11
I20260811 01:57:20.344378 10906 cluster_itest_util.cc:258] Not converged past 0 yet: 10.10 11.11
I20260811 01:57:20.455924 11098 raft_consensus.cc:1275] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 11 FOLLOWER]: Refusing update from remote peer bd948c647ec746139d386d7d38522a90: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260811 01:57:20.456348 11498 consensus_queue.cc:1048] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067a4448010f4c4096f6a6881e20ad5e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260811 01:57:20.847015 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:20.851140 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:20.851464 10906 tablet_replica.cc:333] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e: stopping tablet replica
I20260811 01:57:20.851756 10906 raft_consensus.cc:2243] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 11 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:20.851939 10906 raft_consensus.cc:2272] T d2ec78a0139d4f6187d9c006064aa70a P 067a4448010f4c4096f6a6881e20ad5e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:20.853040 10906 tablet_replica.cc:333] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e: stopping tablet replica
I20260811 01:57:20.853133 10906 raft_consensus.cc:2243] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:20.853240 10906 raft_consensus.cc:2272] T fd68c34d935e49f69b64fa5f1c20bb2d P 067a4448010f4c4096f6a6881e20ad5e [term 9 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:20.866346 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:20.869431 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:20.872856 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:20.873041 10906 tablet_replica.cc:333] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90: stopping tablet replica
I20260811 01:57:20.873138 10906 raft_consensus.cc:2243] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 11 LEADER]: Raft consensus shutting down.
I20260811 01:57:20.873297 10906 raft_consensus.cc:2272] T d2ec78a0139d4f6187d9c006064aa70a P bd948c647ec746139d386d7d38522a90 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:20.873767 10906 tablet_replica.cc:333] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90: stopping tablet replica
I20260811 01:57:20.873847 10906 raft_consensus.cc:2243] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 9 LEADER]: Raft consensus shutting down.
I20260811 01:57:20.873984 10906 raft_consensus.cc:2272] T fd68c34d935e49f69b64fa5f1c20bb2d P bd948c647ec746139d386d7d38522a90 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:20.886236 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:20.888015 10906 master.cc:562] Master@127.10.166.190:43307 shutting down...
I20260811 01:57:20.890772 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:20.890893 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:20.890944 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 843d2c61fc2342cab12a05d74c97a092: stopping tablet replica
I20260811 01:57:20.902951 10906 master.cc:584] Master@127.10.166.190:43307 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (14458 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260811 01:57:20.908974 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:42459
I20260811 01:57:20.909269 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260811 01:57:20.910871 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:20.910996 11533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:20.910960 11535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:20.910977 11532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:20.911343 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:20.911388 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:20.911409 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413440911408 us; error 0 us; skew 500 ppm
I20260811 01:57:20.915441 10906 webserver.cc:533] Webserver started at http://127.10.166.190:33697/ using document root <none> and password file <none>
I20260811 01:57:20.915589 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:20.915640 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:20.915745 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:20.916116 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "de2f34e2b8cc4384b3b6bcd5052122c0"
format_stamp: "Formatted at 2026-08-11 01:57:20 on dist-test-slave-1l3l"
I20260811 01:57:20.917586 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:20.918421 11546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:20.918618 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:20.918686 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "de2f34e2b8cc4384b3b6bcd5052122c0"
format_stamp: "Formatted at 2026-08-11 01:57:20 on dist-test-slave-1l3l"
I20260811 01:57:20.918752 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:20.941540 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:20.941864 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:20.946668 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:42459
I20260811 01:57:20.948848 11619 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:42459 every 8 connection(s)
I20260811 01:57:20.949323 11622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:20.951337 11622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: Bootstrap starting.
I20260811 01:57:20.952142 11622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:20.953243 11622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: No bootstrap required, opened a new log
I20260811 01:57:20.953701 11622 raft_consensus.cc:359] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER }
I20260811 01:57:20.953845 11622 raft_consensus.cc:385] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:20.953888 11622 raft_consensus.cc:740] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2f34e2b8cc4384b3b6bcd5052122c0, State: Initialized, Role: FOLLOWER
I20260811 01:57:20.954031 11622 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER }
I20260811 01:57:20.954116 11622 raft_consensus.cc:399] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:20.954151 11622 raft_consensus.cc:493] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:20.954198 11622 raft_consensus.cc:3060] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:20.955080 11622 raft_consensus.cc:515] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER }
I20260811 01:57:20.955209 11622 leader_election.cc:304] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de2f34e2b8cc4384b3b6bcd5052122c0; no voters:
I20260811 01:57:20.955427 11622 leader_election.cc:290] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:20.955557 11625 raft_consensus.cc:2804] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:20.955838 11625 raft_consensus.cc:697] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 1 LEADER]: Becoming Leader. State: Replica: de2f34e2b8cc4384b3b6bcd5052122c0, State: Running, Role: LEADER
I20260811 01:57:20.955899 11622 sys_catalog.cc:565] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:20.956059 11625 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER }
I20260811 01:57:20.956488 11626 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER } }
I20260811 01:57:20.956576 11626 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:20.956773 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2f34e2b8cc4384b3b6bcd5052122c0. Latest consensus state: current_term: 1 leader_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2f34e2b8cc4384b3b6bcd5052122c0" member_type: VOTER } }
I20260811 01:57:20.956836 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:20.957839 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20260811 01:57:20.958618 11649 catalog_manager.cc:1594] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:57:20.958693 11649 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:57:20.958775 11635 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:20.959475 11635 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:20.964148 11635 catalog_manager.cc:1383] Generated new cluster ID: 6267f49fe23f40939ad81308f179349a
I20260811 01:57:20.964208 11635 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:20.995518 11635 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:20.996137 11635 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:21.019438 11635 catalog_manager.cc:6092] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: Generated new TSK 0
I20260811 01:57:21.019645 11635 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:21.022223 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:21.024010 11656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:21.024145 11651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:21.024168 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:21.024299 11653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:21.024511 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:21.024565 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:21.024586 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413441024586 us; error 0 us; skew 500 ppm
I20260811 01:57:21.025408 10906 webserver.cc:533] Webserver started at http://127.10.166.129:41053/ using document root <none> and password file <none>
I20260811 01:57:21.025579 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:21.025633 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:21.025707 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:21.026100 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "916c97cb9d0149d5ad39111cd8839e58"
format_stamp: "Formatted at 2026-08-11 01:57:21 on dist-test-slave-1l3l"
I20260811 01:57:21.027580 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:21.028764 11665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.029068 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:21.029147 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "916c97cb9d0149d5ad39111cd8839e58"
format_stamp: "Formatted at 2026-08-11 01:57:21 on dist-test-slave-1l3l"
I20260811 01:57:21.029222 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:21.036197 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:21.036499 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:21.036727 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:21.037149 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:21.037185 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.037218 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:21.037245 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.041239 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:35391
I20260811 01:57:21.041297 11761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:35391 every 8 connection(s)
I20260811 01:57:21.042047 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:21.043613 11771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:21.043819 11775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:21.043880 11779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:21.043970 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:21.044158 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:21.044201 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:21.044215 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413441044216 us; error 0 us; skew 500 ppm
I20260811 01:57:21.045073 10906 webserver.cc:533] Webserver started at http://127.10.166.130:46863/ using document root <none> and password file <none>
I20260811 01:57:21.045233 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:21.045284 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:21.045346 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:21.045733 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "4b05e2f576294b05ad3cdbe888bc6ef9"
format_stamp: "Formatted at 2026-08-11 01:57:21 on dist-test-slave-1l3l"
I20260811 01:57:21.046653 11763 heartbeater.cc:344] Connected to a master server at 127.10.166.190:42459
I20260811 01:57:21.046737 11763 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:21.046941 11763 heartbeater.cc:507] Master 127.10.166.190:42459 requested a full tablet report, sending...
I20260811 01:57:21.047287 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:21.047546 11565 ts_manager.cc:194] Registered new tserver with Master: 916c97cb9d0149d5ad39111cd8839e58 (127.10.166.129:35391)
I20260811 01:57:21.048264 11784 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.048283 11565 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37370
I20260811 01:57:21.048501 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:21.048570 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "4b05e2f576294b05ad3cdbe888bc6ef9"
format_stamp: "Formatted at 2026-08-11 01:57:21 on dist-test-slave-1l3l"
I20260811 01:57:21.048638 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:21.066766 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:21.067157 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:21.067456 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:21.067956 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:21.067996 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.068041 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:21.068070 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:21.072610 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:35597
I20260811 01:57:21.072675 11888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:35597 every 8 connection(s)
I20260811 01:57:21.076941 11891 heartbeater.cc:344] Connected to a master server at 127.10.166.190:42459
I20260811 01:57:21.077039 11891 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:21.077247 11891 heartbeater.cc:507] Master 127.10.166.190:42459 requested a full tablet report, sending...
I20260811 01:57:21.077880 11565 ts_manager.cc:194] Registered new tserver with Master: 4b05e2f576294b05ad3cdbe888bc6ef9 (127.10.166.130:35597)
I20260811 01:57:21.078370 11565 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37376
I20260811 01:57:21.078750 10906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.005726948s
I20260811 01:57:21.085103 11565 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:57:21.085510 11565 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260811 01:57:21.093538 11702 tablet_service.cc:1511] Processing CreateTablet for tablet 127e92538f504c0395c838f870343d52 (DEFAULT_TABLE table=test-table [id=d660e0e01ba04e928b5014679e7b328d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:21.093817 11702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e92538f504c0395c838f870343d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:21.093793 11822 tablet_service.cc:1511] Processing CreateTablet for tablet 127e92538f504c0395c838f870343d52 (DEFAULT_TABLE table=test-table [id=d660e0e01ba04e928b5014679e7b328d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:21.094039 11822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e92538f504c0395c838f870343d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:21.095770 11907 tablet_bootstrap.cc:492] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: Bootstrap starting.
I20260811 01:57:21.096037 11908 tablet_bootstrap.cc:492] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: Bootstrap starting.
I20260811 01:57:21.096707 11907 tablet_bootstrap.cc:654] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:21.096895 11908 tablet_bootstrap.cc:654] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:21.097700 11907 tablet_bootstrap.cc:492] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: No bootstrap required, opened a new log
I20260811 01:57:21.097782 11907 ts_tablet_manager.cc:1403] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:21.097848 11908 tablet_bootstrap.cc:492] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: No bootstrap required, opened a new log
I20260811 01:57:21.097934 11908 ts_tablet_manager.cc:1403] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:21.098210 11907 raft_consensus.cc:359] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.098297 11907 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Initialized, Role: FOLLOWER
I20260811 01:57:21.098310 11908 raft_consensus.cc:359] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.098417 11908 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Initialized, Role: FOLLOWER
I20260811 01:57:21.098397 11907 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.098520 11908 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.098744 11907 ts_tablet_manager.cc:1434] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:21.098798 11763 heartbeater.cc:499] Master 127.10.166.190:42459 was elected leader, sending a full tablet report...
I20260811 01:57:21.098881 11908 ts_tablet_manager.cc:1434] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:21.099020 11891 heartbeater.cc:499] Master 127.10.166.190:42459 was elected leader, sending a full tablet report...
I20260811 01:57:21.109982 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:21.110038 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 0 FOLLOWER]: Emulating election...
I20260811 01:57:21.110088 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:21.110850 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 1 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
W20260811 01:57:21.111025 11891 heartbeater.cc:646] Failed to heartbeat to 127.10.166.190:42459 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260811 01:57:21.111009 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.111596 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:21.118896 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:21.220757 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:21.422721 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 0.0 1.1
I20260811 01:57:21.523912 11718 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:21.525331 11916 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:21.725695 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:21.725871 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 1 FOLLOWER]: Emulating election...
I20260811 01:57:21.725953 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:21.727072 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:21.727242 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.727795 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:21.729094 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:21.830796 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:21.962157 11718 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 LEADER]: Rejecting Update request from peer 4b05e2f576294b05ad3cdbe888bc6ef9 for earlier term 1. Current term is 2. Ops: []
I20260811 01:57:21.962622 11923 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:21.962819 11923 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer 916c97cb9d0149d5ad39111cd8839e58 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260811 01:57:21.962884 11916 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:21.962965 11916 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:21.963089 11916 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:21.963223 11916 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:22.032618 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 2.2 1.1
I20260811 01:57:22.266853 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 2 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 01:57:22.267302 11926 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:22.335491 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:22.335567 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 LEADER]: Emulating election...
I20260811 01:57:22.335606 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:22.335640 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:22.335781 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:22.335934 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:22.336827 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 3 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:22.336995 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:22.337469 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:22.338742 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:22.440563 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:22.642395 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 2.2
I20260811 01:57:22.894392 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:22.895818 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 3 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:22.896292 11933 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:22.945226 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:22.945302 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 3 FOLLOWER]: Emulating election...
I20260811 01:57:22.945359 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:22.946261 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:22.946457 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:22.946895 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:22.948184 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:23.049845 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:23.251668 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 3.3 4.4
I20260811 01:57:23.282141 11835 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 LEADER]: Rejecting Update request from peer 916c97cb9d0149d5ad39111cd8839e58 for earlier term 3. Current term is 4. Ops: []
I20260811 01:57:23.282648 11933 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:23.282809 11933 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer 4b05e2f576294b05ad3cdbe888bc6ef9 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260811 01:57:23.282912 11935 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 01:57:23.282981 11935 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:23.283093 11935 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:23.283252 11935 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:23.472935 11718 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 4 FOLLOWER]: Refusing update from remote peer 4b05e2f576294b05ad3cdbe888bc6ef9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260811 01:57:23.473461 11938 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:23.554651 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:23.554750 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 LEADER]: Emulating election...
I20260811 01:57:23.554808 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 LEADER]: Stepping down as leader of term 4
I20260811 01:57:23.554847 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:23.554975 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:23.555164 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:23.556492 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 5 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:23.556659 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:23.557164 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:23.558327 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:23.660044 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:23.861830 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 4.4 5.5
I20260811 01:57:23.958765 11718 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:23.959830 11718 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 5 FOLLOWER]: Refusing update from remote peer 4b05e2f576294b05ad3cdbe888bc6ef9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260811 01:57:23.960300 11944 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:24.164840 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:24.164937 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 5 LEADER]: Emulating election...
I20260811 01:57:24.164994 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 5 LEADER]: Stepping down as leader of term 5
I20260811 01:57:24.165030 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:24.165153 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:24.165309 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:24.166236 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 6 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:24.166424 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:24.166844 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:24.168066 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 6.6
I20260811 01:57:24.269884 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 6.6
I20260811 01:57:24.471683 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 5.5 6.6
I20260811 01:57:24.589942 11718 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:24.590974 11718 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 6 FOLLOWER]: Refusing update from remote peer 4b05e2f576294b05ad3cdbe888bc6ef9: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260811 01:57:24.591488 11948 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:24.774863 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:24.774960 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 6 FOLLOWER]: Emulating election...
I20260811 01:57:24.775017 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:24.776214 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:24.776398 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:24.776854 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:24.778074 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 6.6
I20260811 01:57:24.879997 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 6.6
I20260811 01:57:25.081845 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 7.7 6.6
I20260811 01:57:25.103073 11718 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 LEADER]: Rejecting Update request from peer 4b05e2f576294b05ad3cdbe888bc6ef9 for earlier term 6. Current term is 7. Ops: []
I20260811 01:57:25.103624 11957 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:25.103938 11957 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer 916c97cb9d0149d5ad39111cd8839e58 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260811 01:57:25.104030 11957 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 6 LEADER]: Stepping down as leader of term 6
I20260811 01:57:25.104079 11957 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:25.104192 11957 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:25.104350 11957 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:25.273638 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 7 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260811 01:57:25.274122 11949 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:25.384843 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:25.384948 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 LEADER]: Emulating election...
I20260811 01:57:25.385002 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 LEADER]: Stepping down as leader of term 7
I20260811 01:57:25.385037 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:25.385141 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:25.385283 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:25.386173 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 8 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:25.386320 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:25.386799 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:25.388115 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 7.7
I20260811 01:57:25.489981 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 7.7
I20260811 01:57:25.691725 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 8.8 7.7
I20260811 01:57:25.857438 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:25.858822 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 8 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 01:57:25.859303 11949 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:57:25.994740 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:25.994827 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 8 LEADER]: Emulating election...
I20260811 01:57:25.994882 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 8 LEADER]: Stepping down as leader of term 8
I20260811 01:57:25.994920 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:25.995066 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:25.995227 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:25.996189 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 9 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:25.996359 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:25.996806 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:25.998070 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 8.8
I20260811 01:57:26.099852 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 8.8
I20260811 01:57:26.301750 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 8.8
I20260811 01:57:26.391353 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:26.392465 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 9 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260811 01:57:26.392930 11949 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 01:57:26.604730 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:26.604835 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 9 FOLLOWER]: Emulating election...
I20260811 01:57:26.604888 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:26.606593 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 10 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:26.606777 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:26.607296 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:26.608672 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 10.10
I20260811 01:57:26.710520 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 10.10
I20260811 01:57:26.912340 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 9.9 10.10
I20260811 01:57:26.962705 11835 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 10 LEADER]: Rejecting Update request from peer 916c97cb9d0149d5ad39111cd8839e58 for earlier term 9. Current term is 10. Ops: []
I20260811 01:57:26.963194 11974 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 01:57:26.963440 11974 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer 4b05e2f576294b05ad3cdbe888bc6ef9 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260811 01:57:26.963553 11979 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 9 LEADER]: Stepping down as leader of term 9
I20260811 01:57:26.963637 11979 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:26.963780 11979 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:26.963961 11979 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:27.178498 11718 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 10 FOLLOWER]: Refusing update from remote peer 4b05e2f576294b05ad3cdbe888bc6ef9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260811 01:57:27.179046 11980 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 01:57:27.215090 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:27.215178 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 10 FOLLOWER]: Emulating election...
I20260811 01:57:27.215217 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 10 FOLLOWER]: Advancing to term 11
I20260811 01:57:27.216356 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:27.216535 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:27.216962 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:27.218179 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 11.11 10.10
I20260811 01:57:27.319969 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 11.11 10.10
I20260811 01:57:27.521833 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 11.11 10.10
I20260811 01:57:27.598367 11718 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 LEADER]: Rejecting Update request from peer 4b05e2f576294b05ad3cdbe888bc6ef9 for earlier term 10. Current term is 11. Ops: []
I20260811 01:57:27.598874 11980 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260811 01:57:27.599064 11980 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer 916c97cb9d0149d5ad39111cd8839e58 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260811 01:57:27.599131 11981 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 10 LEADER]: Stepping down as leader of term 10
I20260811 01:57:27.599210 11981 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:27.599323 11981 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:27.599470 11981 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 10 FOLLOWER]: Advancing to term 11
I20260811 01:57:27.760277 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 11 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260811 01:57:27.760788 11984 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260811 01:57:27.824673 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:27.824761 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 LEADER]: Emulating election...
I20260811 01:57:27.824802 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 LEADER]: Stepping down as leader of term 11
I20260811 01:57:27.824839 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:27.824977 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:27.825132 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 11 FOLLOWER]: Advancing to term 12
I20260811 01:57:27.826025 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 12 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:27.826176 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:27.826608 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:27.828022 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 12.12 11.11
I20260811 01:57:27.929780 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 12.12 11.11
I20260811 01:57:28.131652 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 12.12 11.11
I20260811 01:57:28.390648 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 11 FOLLOWER]: Advancing to term 12
I20260811 01:57:28.392069 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 12 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260811 01:57:28.392596 11989 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260811 01:57:28.434433 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:28.434509 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 12 LEADER]: Emulating election...
I20260811 01:57:28.434557 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 12 LEADER]: Stepping down as leader of term 12
I20260811 01:57:28.434592 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:28.434702 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:28.434856 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 12 FOLLOWER]: Advancing to term 13
I20260811 01:57:28.435766 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 13 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:28.435914 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:28.436343 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:28.437568 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 12.12
I20260811 01:57:28.539302 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 12.12
I20260811 01:57:28.741130 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 12.12
I20260811 01:57:28.872109 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 12 FOLLOWER]: Advancing to term 13
I20260811 01:57:28.873179 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 13 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260811 01:57:28.873744 11989 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 01:57:29.044037 11918 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:29.044117 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 13 FOLLOWER]: Emulating election...
I20260811 01:57:29.044176 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 13 FOLLOWER]: Advancing to term 14
I20260811 01:57:29.045089 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 14 LEADER]: Becoming Leader. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:29.045270 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:29.045746 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:29.046993 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 14.14
I20260811 01:57:29.148792 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 14.14
I20260811 01:57:29.285537 11835 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 14 LEADER]: Rejecting Update request from peer 916c97cb9d0149d5ad39111cd8839e58 for earlier term 13. Current term is 14. Ops: []
I20260811 01:57:29.286085 11989 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260811 01:57:29.286267 11989 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Peer 4b05e2f576294b05ad3cdbe888bc6ef9 log is divergent from this leader: its last log entry 14.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260811 01:57:29.286360 11991 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 13 LEADER]: Stepping down as leader of term 13
I20260811 01:57:29.286422 11991 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:29.286526 11991 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:29.286649 11991 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 13 FOLLOWER]: Advancing to term 14
I20260811 01:57:29.350586 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 13.13 14.14
I20260811 01:57:29.600540 11718 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 14 FOLLOWER]: Refusing update from remote peer 4b05e2f576294b05ad3cdbe888bc6ef9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260811 01:57:29.601109 11996 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260811 01:57:29.653664 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:29.653750 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 14 FOLLOWER]: Emulating election...
I20260811 01:57:29.653803 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 14 FOLLOWER]: Advancing to term 15
I20260811 01:57:29.654749 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:29.654942 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:29.655469 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:29.656813 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 15.15 14.14
I20260811 01:57:29.758575 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 15.15 14.14
I20260811 01:57:29.960436 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 15.15 14.14
I20260811 01:57:30.000262 11718 raft_consensus.cc:1240] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 LEADER]: Rejecting Update request from peer 4b05e2f576294b05ad3cdbe888bc6ef9 for earlier term 14. Current term is 15. Ops: []
I20260811 01:57:30.000767 11997 consensus_queue.cc:1059] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260811 01:57:30.000962 11997 consensus_queue.cc:1243] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [LEADER]: Peer 916c97cb9d0149d5ad39111cd8839e58 log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260811 01:57:30.001027 11996 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 14 LEADER]: Stepping down as leader of term 14
I20260811 01:57:30.001083 11996 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b05e2f576294b05ad3cdbe888bc6ef9, State: Running, Role: LEADER
I20260811 01:57:30.001204 11996 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:30.001338 11996 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 14 FOLLOWER]: Advancing to term 15
I20260811 01:57:30.262297 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 15.15 14.14
I20260811 01:57:30.281425 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 15 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260811 01:57:30.281967 12002 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260811 01:57:30.665486 11918 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:30.665586 11918 raft_consensus.cc:423] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 LEADER]: Emulating election...
I20260811 01:57:30.665652 11918 raft_consensus.cc:3055] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 LEADER]: Stepping down as leader of term 15
I20260811 01:57:30.665688 11918 raft_consensus.cc:740] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:30.665820 11918 consensus_queue.cc:260] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:30.665992 11918 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 15 FOLLOWER]: Advancing to term 16
I20260811 01:57:30.666957 11918 raft_consensus.cc:697] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 16 LEADER]: Becoming Leader. State: Replica: 916c97cb9d0149d5ad39111cd8839e58, State: Running, Role: LEADER
I20260811 01:57:30.667140 11918 consensus_queue.cc:237] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916c97cb9d0149d5ad39111cd8839e58" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 35391 } } peers { permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 } }
I20260811 01:57:30.667609 11918 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:30.668915 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 16.16 15.15
I20260811 01:57:30.770736 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 16.16 15.15
I20260811 01:57:30.972590 11918 cluster_itest_util.cc:258] Not converged past 0 yet: 16.16 15.15
I20260811 01:57:31.062917 11835 raft_consensus.cc:3060] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 15 FOLLOWER]: Advancing to term 16
I20260811 01:57:31.064261 11835 raft_consensus.cc:1275] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 16 FOLLOWER]: Refusing update from remote peer 916c97cb9d0149d5ad39111cd8839e58: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260811 01:57:31.064747 12003 consensus_queue.cc:1048] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b05e2f576294b05ad3cdbe888bc6ef9" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260811 01:57:31.275030 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:31.278568 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:31.278762 10906 tablet_replica.cc:333] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58: stopping tablet replica
I20260811 01:57:31.278872 10906 raft_consensus.cc:2243] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 16 LEADER]: Raft consensus shutting down.
I20260811 01:57:31.279053 10906 raft_consensus.cc:2272] T 127e92538f504c0395c838f870343d52 P 916c97cb9d0149d5ad39111cd8839e58 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:31.291739 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:31.293509 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:31.296713 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:31.296875 10906 tablet_replica.cc:333] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9: stopping tablet replica
I20260811 01:57:31.296969 10906 raft_consensus.cc:2243] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:31.297080 10906 raft_consensus.cc:2272] T 127e92538f504c0395c838f870343d52 P 4b05e2f576294b05ad3cdbe888bc6ef9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:31.309291 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:31.310925 10906 master.cc:562] Master@127.10.166.190:42459 shutting down...
I20260811 01:57:31.313723 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:31.313854 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:31.313899 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P de2f34e2b8cc4384b3b6bcd5052122c0: stopping tablet replica
I20260811 01:57:31.325660 10906 master.cc:584] Master@127.10.166.190:42459 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (10422 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260811 01:57:31.331240 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:38115
I20260811 01:57:31.331554 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:31.333164 12020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:31.333173 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:31.333176 12022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:31.333341 12025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:31.333518 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:31.333559 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:31.333573 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413451333573 us; error 0 us; skew 500 ppm
I20260811 01:57:31.334261 10906 webserver.cc:533] Webserver started at http://127.10.166.190:41539/ using document root <none> and password file <none>
I20260811 01:57:31.334388 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:31.334424 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:31.334466 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:31.334856 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "289bc10bb1c043cfa9e6fd46fb7062d0"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.336325 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:31.337145 12033 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.337383 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:31.337450 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "289bc10bb1c043cfa9e6fd46fb7062d0"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.337523 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:31.347078 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:31.347347 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:31.351001 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:38115
I20260811 01:57:31.352389 12113 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:38115 every 8 connection(s)
I20260811 01:57:31.352798 12114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:31.354378 12114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0: Bootstrap starting.
I20260811 01:57:31.355096 12114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:31.356045 12114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0: No bootstrap required, opened a new log
I20260811 01:57:31.356388 12114 raft_consensus.cc:359] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER }
I20260811 01:57:31.356462 12114 raft_consensus.cc:385] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:31.356485 12114 raft_consensus.cc:740] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bc10bb1c043cfa9e6fd46fb7062d0, State: Initialized, Role: FOLLOWER
I20260811 01:57:31.356582 12114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER }
I20260811 01:57:31.356633 12114 raft_consensus.cc:399] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:31.356658 12114 raft_consensus.cc:493] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:31.356689 12114 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:31.357277 12114 raft_consensus.cc:515] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER }
I20260811 01:57:31.357398 12114 leader_election.cc:304] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 289bc10bb1c043cfa9e6fd46fb7062d0; no voters:
I20260811 01:57:31.357532 12114 leader_election.cc:290] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:31.357643 12118 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:31.357853 12118 raft_consensus.cc:697] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 1 LEADER]: Becoming Leader. State: Replica: 289bc10bb1c043cfa9e6fd46fb7062d0, State: Running, Role: LEADER
I20260811 01:57:31.357944 12114 sys_catalog.cc:565] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:31.357997 12118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER }
I20260811 01:57:31.358402 12120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER } }
I20260811 01:57:31.358430 12121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 289bc10bb1c043cfa9e6fd46fb7062d0. Latest consensus state: current_term: 1 leader_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "289bc10bb1c043cfa9e6fd46fb7062d0" member_type: VOTER } }
I20260811 01:57:31.358487 12120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:31.358512 12121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:31.359094 12125 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:31.360057 12125 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:31.360234 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:31.361775 12125 catalog_manager.cc:1383] Generated new cluster ID: fdfa04a958e944fb8b151e8523f76a91
I20260811 01:57:31.361832 12125 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:31.379765 12125 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:31.380287 12125 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:31.385987 12125 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0: Generated new TSK 0
I20260811 01:57:31.386132 12125 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:31.392344 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:31.393972 12148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:31.394045 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:31.394083 12150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:31.393973 12146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:31.394375 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:31.394419 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:31.394447 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413451394447 us; error 0 us; skew 500 ppm
I20260811 01:57:31.395181 10906 webserver.cc:533] Webserver started at http://127.10.166.129:45687/ using document root <none> and password file <none>
I20260811 01:57:31.395334 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:31.395380 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:31.395462 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:31.395869 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "be36fcfd62e94623b555801a7b6620cf"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.397212 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:31.398037 12158 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.398243 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:31.398304 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "be36fcfd62e94623b555801a7b6620cf"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.398370 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:31.432170 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:31.432598 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:31.432888 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:31.433390 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:31.433447 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.433487 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:31.433514 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.437508 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:34085
I20260811 01:57:31.437568 12251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:34085 every 8 connection(s)
I20260811 01:57:31.438340 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:31.440092 12259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:31.440056 12258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:31.440048 12261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:31.440412 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:31.440595 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:31.440634 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:31.440653 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413451440653 us; error 0 us; skew 500 ppm
I20260811 01:57:31.441514 10906 webserver.cc:533] Webserver started at http://127.10.166.130:35325/ using document root <none> and password file <none>
I20260811 01:57:31.441671 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:31.441721 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:31.441790 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:31.442232 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "35b50e28fbfd453894c2fdf533af7a7a"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.442786 12252 heartbeater.cc:344] Connected to a master server at 127.10.166.190:38115
I20260811 01:57:31.442876 12252 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:31.443084 12252 heartbeater.cc:507] Master 127.10.166.190:38115 requested a full tablet report, sending...
I20260811 01:57:31.443733 12059 ts_manager.cc:194] Registered new tserver with Master: be36fcfd62e94623b555801a7b6620cf (127.10.166.129:34085)
I20260811 01:57:31.443919 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:31.444464 12059 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49272
I20260811 01:57:31.444862 12270 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.445101 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:31.445168 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "35b50e28fbfd453894c2fdf533af7a7a"
format_stamp: "Formatted at 2026-08-11 01:57:31 on dist-test-slave-1l3l"
I20260811 01:57:31.445233 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:31.462141 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:31.462435 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:31.462684 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:31.463080 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:31.463116 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.463155 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:31.463183 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:31.467442 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:35593
I20260811 01:57:31.467495 12370 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:35593 every 8 connection(s)
I20260811 01:57:31.471247 12371 heartbeater.cc:344] Connected to a master server at 127.10.166.190:38115
I20260811 01:57:31.471328 12371 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:31.471517 12371 heartbeater.cc:507] Master 127.10.166.190:38115 requested a full tablet report, sending...
I20260811 01:57:31.472088 12059 ts_manager.cc:194] Registered new tserver with Master: 35b50e28fbfd453894c2fdf533af7a7a (127.10.166.130:35593)
I20260811 01:57:31.472333 10906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.004481158s
I20260811 01:57:31.472699 12059 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49288
I20260811 01:57:31.478518 12059 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:57:31.478907 12059 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260811 01:57:31.486685 12199 tablet_service.cc:1511] Processing CreateTablet for tablet a7e484e0bacf4470ae96d5d1091cee36 (DEFAULT_TABLE table=test-table [id=8f4c8bd2b5874655959099f3d7288475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:31.486955 12199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e484e0bacf4470ae96d5d1091cee36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:31.487485 12317 tablet_service.cc:1511] Processing CreateTablet for tablet a7e484e0bacf4470ae96d5d1091cee36 (DEFAULT_TABLE table=test-table [id=8f4c8bd2b5874655959099f3d7288475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:31.487751 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e484e0bacf4470ae96d5d1091cee36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:31.489768 12389 tablet_bootstrap.cc:492] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: Bootstrap starting.
I20260811 01:57:31.489804 12388 tablet_bootstrap.cc:492] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: Bootstrap starting.
I20260811 01:57:31.490633 12388 tablet_bootstrap.cc:654] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:31.490646 12389 tablet_bootstrap.cc:654] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:31.491577 12388 tablet_bootstrap.cc:492] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: No bootstrap required, opened a new log
I20260811 01:57:31.491619 12389 tablet_bootstrap.cc:492] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: No bootstrap required, opened a new log
I20260811 01:57:31.491658 12388 ts_tablet_manager.cc:1403] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:31.491686 12389 ts_tablet_manager.cc:1403] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:31.492156 12388 raft_consensus.cc:359] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:31.492296 12388 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Initialized, Role: FOLLOWER
I20260811 01:57:31.492270 12389 raft_consensus.cc:359] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:31.492368 12389 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Initialized, Role: FOLLOWER
I20260811 01:57:31.492410 12388 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:31.492473 12389 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:31.492761 12252 heartbeater.cc:499] Master 127.10.166.190:38115 was elected leader, sending a full tablet report...
I20260811 01:57:31.492802 12371 heartbeater.cc:499] Master 127.10.166.190:38115 was elected leader, sending a full tablet report...
I20260811 01:57:31.492827 12389 ts_tablet_manager.cc:1434] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:31.492761 12388 ts_tablet_manager.cc:1434] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:31.501158 10906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:31.501209 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 0 FOLLOWER]: Emulating election...
I20260811 01:57:31.501237 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:31.502008 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 1 LEADER]: Becoming Leader. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:31.502133 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:31.502617 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:31.508616 10906 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1
I20260811 01:57:31.610332 10906 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1
I20260811 01:57:31.812196 10906 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1
I20260811 01:57:32.030094 12212 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:32.032971 12399 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:32.114316 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:32.114390 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 1 FOLLOWER]: Emulating election...
I20260811 01:57:32.114423 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:32.115330 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:32.115518 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:32.116074 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:32.117264 10906 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260811 01:57:32.218988 10906 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260811 01:57:32.420742 10906 cluster_itest_util.cc:258] Not converged past 2 yet: 2.2 1.1
I20260811 01:57:32.504706 12212 raft_consensus.cc:1240] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 LEADER]: Rejecting Update request from peer 35b50e28fbfd453894c2fdf533af7a7a for earlier term 1. Current term is 2. Ops: []
I20260811 01:57:32.505174 12399 consensus_queue.cc:1059] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:32.505335 12399 consensus_queue.cc:1243] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Peer be36fcfd62e94623b555801a7b6620cf log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260811 01:57:32.505384 12409 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:32.505452 12409 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:32.505577 12409 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:32.505738 12409 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:32.615118 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 2 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 01:57:32.615597 12405 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:32.723384 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:32.723462 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 LEADER]: Emulating election...
I20260811 01:57:32.723507 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:32.723541 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:32.723652 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:32.723845 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:32.724730 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 3 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:32.724892 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:32.725333 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:32.726580 10906 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260811 01:57:32.828547 10906 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260811 01:57:33.030380 10906 cluster_itest_util.cc:258] Not converged past 3 yet: 3.3 2.2
I20260811 01:57:33.124996 12332 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:33.126092 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 3 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:33.126519 12405 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:33.332494 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:33.332571 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 3 LEADER]: Emulating election...
I20260811 01:57:33.332602 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 3 LEADER]: Stepping down as leader of term 3
I20260811 01:57:33.332620 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:33.332700 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:33.332830 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:33.333756 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 4 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:33.333907 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:33.334347 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:33.335510 10906 cluster_itest_util.cc:258] Not converged past 4 yet: 4.4 3.3
I20260811 01:57:33.437333 10906 cluster_itest_util.cc:258] Not converged past 4 yet: 4.4 3.3
I20260811 01:57:33.639164 10906 cluster_itest_util.cc:258] Not converged past 4 yet: 4.4 3.3
I20260811 01:57:33.896217 12332 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:33.897282 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 4 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260811 01:57:33.897756 12431 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:33.940990 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:33.941059 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 4 LEADER]: Emulating election...
I20260811 01:57:33.941100 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 4 LEADER]: Stepping down as leader of term 4
I20260811 01:57:33.941133 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:33.941244 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:33.941387 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:33.942250 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 5 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:33.942392 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:33.942817 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:33.944001 10906 cluster_itest_util.cc:258] Not converged past 5 yet: 5.5 4.4
I20260811 01:57:34.045714 10906 cluster_itest_util.cc:258] Not converged past 5 yet: 5.5 4.4
I20260811 01:57:34.247514 10906 cluster_itest_util.cc:258] Not converged past 5 yet: 5.5 4.4
I20260811 01:57:34.448383 12332 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:34.449345 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 5 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260811 01:57:34.449784 12439 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:34.549507 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:34.549585 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 5 LEADER]: Emulating election...
I20260811 01:57:34.549615 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 5 LEADER]: Stepping down as leader of term 5
I20260811 01:57:34.549634 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:34.549716 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:34.549849 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:34.550707 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 6 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:34.550856 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:34.551278 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:34.552532 10906 cluster_itest_util.cc:258] Not converged past 6 yet: 6.6 5.5
I20260811 01:57:34.654254 10906 cluster_itest_util.cc:258] Not converged past 6 yet: 6.6 5.5
I20260811 01:57:34.855997 10906 cluster_itest_util.cc:258] Not converged past 6 yet: 6.6 5.5
I20260811 01:57:35.110306 12332 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 5 FOLLOWER]: Advancing to term 6
I20260811 01:57:35.111315 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 6 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260811 01:57:35.111804 12452 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:35.158036 10906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:35.158109 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 6 FOLLOWER]: Emulating election...
I20260811 01:57:35.158138 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:35.159022 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 LEADER]: Becoming Leader. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:35.159199 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:35.159639 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:35.160945 10906 cluster_itest_util.cc:258] Not converged past 7 yet: 6.6 7.7
I20260811 01:57:35.262598 10906 cluster_itest_util.cc:258] Not converged past 7 yet: 6.6 7.7
I20260811 01:57:35.464476 10906 cluster_itest_util.cc:258] Not converged past 7 yet: 6.6 7.7
I20260811 01:57:35.492980 12332 raft_consensus.cc:1240] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 LEADER]: Rejecting Update request from peer be36fcfd62e94623b555801a7b6620cf for earlier term 6. Current term is 7. Ops: []
I20260811 01:57:35.493445 12452 consensus_queue.cc:1059] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:35.493606 12452 consensus_queue.cc:1243] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Peer 35b50e28fbfd453894c2fdf533af7a7a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260811 01:57:35.493673 12456 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 6 LEADER]: Stepping down as leader of term 6
I20260811 01:57:35.493727 12456 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:35.493832 12456 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:35.493973 12456 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 6 FOLLOWER]: Advancing to term 7
I20260811 01:57:35.715603 12212 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 7 FOLLOWER]: Refusing update from remote peer 35b50e28fbfd453894c2fdf533af7a7a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260811 01:57:35.716106 12462 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:35.766655 10906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20260811 01:57:35.766732 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 LEADER]: Emulating election...
I20260811 01:57:35.766763 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 LEADER]: Stepping down as leader of term 7
I20260811 01:57:35.766808 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:35.766916 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:35.767077 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:35.768031 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 8 LEADER]: Becoming Leader. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:35.768184 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:35.768631 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:35.769857 10906 cluster_itest_util.cc:258] Not converged past 8 yet: 7.7 8.8
I20260811 01:57:35.871829 10906 cluster_itest_util.cc:258] Not converged past 8 yet: 7.7 8.8
I20260811 01:57:36.073786 10906 cluster_itest_util.cc:258] Not converged past 8 yet: 7.7 8.8
I20260811 01:57:36.302634 12212 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 7 FOLLOWER]: Advancing to term 8
I20260811 01:57:36.303663 12212 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 8 FOLLOWER]: Refusing update from remote peer 35b50e28fbfd453894c2fdf533af7a7a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260811 01:57:36.304141 12467 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:57:36.375871 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:36.375948 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 8 FOLLOWER]: Emulating election...
I20260811 01:57:36.375980 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:36.376856 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:36.377020 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:36.377453 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:36.378664 10906 cluster_itest_util.cc:258] Not converged past 9 yet: 9.9 8.8
I20260811 01:57:36.480410 10906 cluster_itest_util.cc:258] Not converged past 9 yet: 9.9 8.8
I20260811 01:57:36.682250 10906 cluster_itest_util.cc:258] Not converged past 9 yet: 9.9 8.8
I20260811 01:57:36.774701 12212 raft_consensus.cc:1240] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 LEADER]: Rejecting Update request from peer 35b50e28fbfd453894c2fdf533af7a7a for earlier term 8. Current term is 9. Ops: []
I20260811 01:57:36.775174 12467 consensus_queue.cc:1059] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 01:57:36.775339 12467 consensus_queue.cc:1243] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [LEADER]: Peer be36fcfd62e94623b555801a7b6620cf log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260811 01:57:36.775410 12469 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 8 LEADER]: Stepping down as leader of term 8
I20260811 01:57:36.775502 12469 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 35b50e28fbfd453894c2fdf533af7a7a, State: Running, Role: LEADER
I20260811 01:57:36.775632 12469 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:36.775816 12469 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 8 FOLLOWER]: Advancing to term 9
I20260811 01:57:36.862231 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 9 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260811 01:57:36.862704 12468 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 01:57:36.984359 10906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20260811 01:57:36.984448 10906 raft_consensus.cc:423] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 LEADER]: Emulating election...
I20260811 01:57:36.984480 10906 raft_consensus.cc:3055] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 LEADER]: Stepping down as leader of term 9
I20260811 01:57:36.984506 10906 raft_consensus.cc:740] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:36.984584 10906 consensus_queue.cc:260] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:36.984714 10906 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:36.985569 10906 raft_consensus.cc:697] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 10 LEADER]: Becoming Leader. State: Replica: be36fcfd62e94623b555801a7b6620cf, State: Running, Role: LEADER
I20260811 01:57:36.985731 10906 consensus_queue.cc:237] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36fcfd62e94623b555801a7b6620cf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34085 } } peers { permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 } }
I20260811 01:57:36.986167 10906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20260811 01:57:36.987373 10906 cluster_itest_util.cc:258] Not converged past 10 yet: 10.10 9.9
I20260811 01:57:37.089128 10906 cluster_itest_util.cc:258] Not converged past 10 yet: 10.10 9.9
I20260811 01:57:37.290946 10906 cluster_itest_util.cc:258] Not converged past 10 yet: 10.10 9.9
I20260811 01:57:37.364765 12332 raft_consensus.cc:3060] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 9 FOLLOWER]: Advancing to term 10
I20260811 01:57:37.365760 12332 raft_consensus.cc:1275] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 10 FOLLOWER]: Refusing update from remote peer be36fcfd62e94623b555801a7b6620cf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260811 01:57:37.366171 12473 consensus_queue.cc:1048] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b50e28fbfd453894c2fdf533af7a7a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 01:57:37.593739 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:37.597518 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:37.597704 10906 tablet_replica.cc:333] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf: stopping tablet replica
I20260811 01:57:37.597829 10906 raft_consensus.cc:2243] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 10 LEADER]: Raft consensus shutting down.
I20260811 01:57:37.597997 10906 raft_consensus.cc:2272] T a7e484e0bacf4470ae96d5d1091cee36 P be36fcfd62e94623b555801a7b6620cf [term 10 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:37.610872 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:37.612694 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:37.616546 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:37.616720 10906 tablet_replica.cc:333] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a: stopping tablet replica
I20260811 01:57:37.616834 10906 raft_consensus.cc:2243] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 10 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:37.616947 10906 raft_consensus.cc:2272] T a7e484e0bacf4470ae96d5d1091cee36 P 35b50e28fbfd453894c2fdf533af7a7a [term 10 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:37.629433 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:37.631102 10906 master.cc:562] Master@127.10.166.190:38115 shutting down...
I20260811 01:57:37.634181 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:37.634318 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:37.634379 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 289bc10bb1c043cfa9e6fd46fb7062d0: stopping tablet replica
I20260811 01:57:37.647083 10906 master.cc:584] Master@127.10.166.190:38115 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (6321 ms)
[----------] 3 tests from LeadershipChangeReportingTest (31201 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260811 01:57:37.652588 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:33919
I20260811 01:57:37.652918 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:37.654583 12493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.654726 12488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.654736 12487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:37.654821 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:37.654999 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:37.655036 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:37.655055 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413457655055 us; error 0 us; skew 500 ppm
I20260811 01:57:37.655942 10906 webserver.cc:533] Webserver started at http://127.10.166.190:38881/ using document root <none> and password file <none>
I20260811 01:57:37.656088 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:37.656132 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:37.656199 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:37.656543 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "57fdade4dc9940b5af0ca6d152bd1b6e"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.657886 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:37.658766 12504 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.658977 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.659039 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "57fdade4dc9940b5af0ca6d152bd1b6e"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.659106 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:37.665596 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:37.665891 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:37.669826 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:33919
I20260811 01:57:37.670975 12586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:33919 every 8 connection(s)
I20260811 01:57:37.671442 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:37.673134 12587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e: Bootstrap starting.
I20260811 01:57:37.673929 12587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:37.674827 12587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e: No bootstrap required, opened a new log
I20260811 01:57:37.675170 12587 raft_consensus.cc:359] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER }
I20260811 01:57:37.675251 12587 raft_consensus.cc:385] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:37.675292 12587 raft_consensus.cc:740] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fdade4dc9940b5af0ca6d152bd1b6e, State: Initialized, Role: FOLLOWER
I20260811 01:57:37.675434 12587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER }
I20260811 01:57:37.675493 12587 raft_consensus.cc:399] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:37.675518 12587 raft_consensus.cc:493] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:37.675551 12587 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:37.676232 12587 raft_consensus.cc:515] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER }
I20260811 01:57:37.676355 12587 leader_election.cc:304] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57fdade4dc9940b5af0ca6d152bd1b6e; no voters:
I20260811 01:57:37.676492 12587 leader_election.cc:290] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:37.676614 12591 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:37.676790 12591 raft_consensus.cc:697] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 1 LEADER]: Becoming Leader. State: Replica: 57fdade4dc9940b5af0ca6d152bd1b6e, State: Running, Role: LEADER
I20260811 01:57:37.676880 12587 sys_catalog.cc:565] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:37.676954 12591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER }
I20260811 01:57:37.677383 12594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57fdade4dc9940b5af0ca6d152bd1b6e. Latest consensus state: current_term: 1 leader_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER } }
I20260811 01:57:37.677371 12592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fdade4dc9940b5af0ca6d152bd1b6e" member_type: VOTER } }
I20260811 01:57:37.677477 12594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:37.677487 12592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:37.677768 12600 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:37.678910 12600 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:37.679064 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:37.680779 12600 catalog_manager.cc:1383] Generated new cluster ID: 356d708e57044664be1db9dca28a3cad
I20260811 01:57:37.680838 12600 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:37.689327 12600 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:37.689862 12600 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:37.697566 12600 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e: Generated new TSK 0
I20260811 01:57:37.697734 12600 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:37.711485 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:37.713367 12617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:37.713428 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:37.713479 12622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.713392 12616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:37.713727 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:37.713774 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:37.713794 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413457713794 us; error 0 us; skew 500 ppm
I20260811 01:57:37.714593 10906 webserver.cc:533] Webserver started at http://127.10.166.129:33855/ using document root <none> and password file <none>
I20260811 01:57:37.714746 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:37.714797 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:37.714869 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:37.715247 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "1e4d42b74842428593e2c2791df6e508"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.716739 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:37.717648 12628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.717859 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:37.717924 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "1e4d42b74842428593e2c2791df6e508"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.718000 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:37.726413 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:37.726743 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:37.727034 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:37.727474 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:37.727522 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.727568 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:37.727588 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.731874 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:39773
I20260811 01:57:37.731914 12735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:39773 every 8 connection(s)
I20260811 01:57:37.732899 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:37.734429 12741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.734481 12742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.734519 12746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:37.734917 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:37.735062 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:37.735097 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:37.735117 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413457735117 us; error 0 us; skew 500 ppm
I20260811 01:57:37.736009 10906 webserver.cc:533] Webserver started at http://127.10.166.130:39917/ using document root <none> and password file <none>
I20260811 01:57:37.736159 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:37.736204 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:37.736277 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:37.736639 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "c52fdc6bd434447699647a8ed1fd1cbe"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.737974 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:37.738788 12757 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.739013 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.739077 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "c52fdc6bd434447699647a8ed1fd1cbe"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.739141 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:37.740988 12736 heartbeater.cc:344] Connected to a master server at 127.10.166.190:33919
I20260811 01:57:37.741083 12736 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:37.741300 12736 heartbeater.cc:507] Master 127.10.166.190:33919 requested a full tablet report, sending...
I20260811 01:57:37.741880 12533 ts_manager.cc:194] Registered new tserver with Master: 1e4d42b74842428593e2c2791df6e508 (127.10.166.129:39773)
I20260811 01:57:37.742589 12533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58144
I20260811 01:57:37.753780 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:37.754091 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:37.754349 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:37.754758 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:37.754796 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.754837 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:37.754865 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.759091 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:34169
I20260811 01:57:37.759142 12856 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:34169 every 8 connection(s)
I20260811 01:57:37.759982 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:37.761722 12861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.761807 12864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:37.761749 12860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:37.762109 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:37.762251 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:37.762296 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:37.762326 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413457762325 us; error 0 us; skew 500 ppm
I20260811 01:57:37.763182 10906 webserver.cc:533] Webserver started at http://127.10.166.131:32949/ using document root <none> and password file <none>
I20260811 01:57:37.763332 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:37.763371 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:37.763434 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:37.763897 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-2-root/instance:
uuid: "6845fd5e9fbd42e88d452eb9865aab49"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.763932 12857 heartbeater.cc:344] Connected to a master server at 127.10.166.190:33919
I20260811 01:57:37.764034 12857 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:37.764218 12857 heartbeater.cc:507] Master 127.10.166.190:33919 requested a full tablet report, sending...
I20260811 01:57:37.764783 12533 ts_manager.cc:194] Registered new tserver with Master: c52fdc6bd434447699647a8ed1fd1cbe (127.10.166.130:34169)
I20260811 01:57:37.765266 12533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58152
I20260811 01:57:37.765496 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:37.766465 12874 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.766670 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.766734 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-2-root
uuid: "6845fd5e9fbd42e88d452eb9865aab49"
format_stamp: "Formatted at 2026-08-11 01:57:37 on dist-test-slave-1l3l"
I20260811 01:57:37.766800 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1786413426438645-10906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:37.789036 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:37.789368 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:37.789637 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:37.790032 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:37.790071 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.790110 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:37.790139 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:37.794463 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.131:40903
I20260811 01:57:37.794519 12979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.131:40903 every 8 connection(s)
I20260811 01:57:37.798210 12980 heartbeater.cc:344] Connected to a master server at 127.10.166.190:33919
I20260811 01:57:37.798291 12980 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:37.798454 12980 heartbeater.cc:507] Master 127.10.166.190:33919 requested a full tablet report, sending...
I20260811 01:57:37.798985 12533 ts_manager.cc:194] Registered new tserver with Master: 6845fd5e9fbd42e88d452eb9865aab49 (127.10.166.131:40903)
I20260811 01:57:37.799285 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004459583s
I20260811 01:57:37.799521 12533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58164
I20260811 01:57:37.805510 12533 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260811 01:57:37.805850 12533 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:57:37.814584 12670 tablet_service.cc:1511] Processing CreateTablet for tablet ec6bda1e441c48fcbd5c3229769fa5cb (DEFAULT_TABLE table=test-table [id=8446e79924744208b5c9a66449def442]), partition=
I20260811 01:57:37.814841 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6bda1e441c48fcbd5c3229769fa5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:37.814930 12920 tablet_service.cc:1511] Processing CreateTablet for tablet ec6bda1e441c48fcbd5c3229769fa5cb (DEFAULT_TABLE table=test-table [id=8446e79924744208b5c9a66449def442]), partition=
I20260811 01:57:37.815155 12920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6bda1e441c48fcbd5c3229769fa5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:37.815793 12805 tablet_service.cc:1511] Processing CreateTablet for tablet ec6bda1e441c48fcbd5c3229769fa5cb (DEFAULT_TABLE table=test-table [id=8446e79924744208b5c9a66449def442]), partition=
I20260811 01:57:37.816009 12805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6bda1e441c48fcbd5c3229769fa5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:37.817229 13001 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: Bootstrap starting.
I20260811 01:57:37.817550 13003 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: Bootstrap starting.
I20260811 01:57:37.818034 13004 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: Bootstrap starting.
I20260811 01:57:37.818256 13001 tablet_bootstrap.cc:654] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:37.818650 13003 tablet_bootstrap.cc:654] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:37.819130 13004 tablet_bootstrap.cc:654] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:37.819428 13001 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: No bootstrap required, opened a new log
I20260811 01:57:37.819515 13001 ts_tablet_manager.cc:1403] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:37.819808 13003 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: No bootstrap required, opened a new log
I20260811 01:57:37.819895 13003 ts_tablet_manager.cc:1403] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:37.819964 13001 raft_consensus.cc:359] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.820060 13001 raft_consensus.cc:385] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:37.820086 13001 raft_consensus.cc:740] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4d42b74842428593e2c2791df6e508, State: Initialized, Role: FOLLOWER
I20260811 01:57:37.820188 13001 consensus_queue.cc:260] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.820317 13003 raft_consensus.cc:359] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.820420 13003 raft_consensus.cc:385] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:37.820456 13003 raft_consensus.cc:740] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6845fd5e9fbd42e88d452eb9865aab49, State: Initialized, Role: FOLLOWER
I20260811 01:57:37.820529 13001 ts_tablet_manager.cc:1434] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.820554 12736 heartbeater.cc:499] Master 127.10.166.190:33919 was elected leader, sending a full tablet report...
I20260811 01:57:37.820575 13003 consensus_queue.cc:260] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.820621 13004 tablet_bootstrap.cc:492] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: No bootstrap required, opened a new log
I20260811 01:57:37.820812 13004 ts_tablet_manager.cc:1403] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:57:37.820926 13003 ts_tablet_manager.cc:1434] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.820956 12980 heartbeater.cc:499] Master 127.10.166.190:33919 was elected leader, sending a full tablet report...
I20260811 01:57:37.821803 13004 raft_consensus.cc:359] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.821900 13004 raft_consensus.cc:385] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:37.821938 13004 raft_consensus.cc:740] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52fdc6bd434447699647a8ed1fd1cbe, State: Initialized, Role: FOLLOWER
I20260811 01:57:37.822053 13004 consensus_queue.cc:260] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.822396 13004 ts_tablet_manager.cc:1434] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:37.822415 12857 heartbeater.cc:499] Master 127.10.166.190:33919 was elected leader, sending a full tablet report...
I20260811 01:57:37.971643 13017 raft_consensus.cc:493] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:37.971814 13017 raft_consensus.cc:515] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.972586 13017 leader_election.cc:290] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4d42b74842428593e2c2791df6e508 (127.10.166.129:39773), c52fdc6bd434447699647a8ed1fd1cbe (127.10.166.130:34169)
I20260811 01:57:37.976222 12682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6bda1e441c48fcbd5c3229769fa5cb" candidate_uuid: "6845fd5e9fbd42e88d452eb9865aab49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4d42b74842428593e2c2791df6e508" is_pre_election: true
I20260811 01:57:37.976428 12682 raft_consensus.cc:2468] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6845fd5e9fbd42e88d452eb9865aab49 in term 0.
I20260811 01:57:37.976447 12819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6bda1e441c48fcbd5c3229769fa5cb" candidate_uuid: "6845fd5e9fbd42e88d452eb9865aab49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" is_pre_election: true
I20260811 01:57:37.976585 12819 raft_consensus.cc:2468] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6845fd5e9fbd42e88d452eb9865aab49 in term 0.
I20260811 01:57:37.976822 12878 leader_election.cc:304] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4d42b74842428593e2c2791df6e508, 6845fd5e9fbd42e88d452eb9865aab49; no voters:
I20260811 01:57:37.977006 13017 raft_consensus.cc:2804] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:37.977058 13017 raft_consensus.cc:493] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:37.977104 13017 raft_consensus.cc:3060] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:37.977980 13017 raft_consensus.cc:515] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.978308 13017 leader_election.cc:290] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4d42b74842428593e2c2791df6e508 (127.10.166.129:39773), c52fdc6bd434447699647a8ed1fd1cbe (127.10.166.130:34169)
I20260811 01:57:37.978621 12682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6bda1e441c48fcbd5c3229769fa5cb" candidate_uuid: "6845fd5e9fbd42e88d452eb9865aab49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4d42b74842428593e2c2791df6e508"
I20260811 01:57:37.978708 12682 raft_consensus.cc:3060] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:37.978659 12819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6bda1e441c48fcbd5c3229769fa5cb" candidate_uuid: "6845fd5e9fbd42e88d452eb9865aab49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52fdc6bd434447699647a8ed1fd1cbe"
I20260811 01:57:37.978817 12819 raft_consensus.cc:3060] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:37.979568 12682 raft_consensus.cc:2468] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6845fd5e9fbd42e88d452eb9865aab49 in term 1.
I20260811 01:57:37.979609 12819 raft_consensus.cc:2468] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6845fd5e9fbd42e88d452eb9865aab49 in term 1.
I20260811 01:57:37.979844 12878 leader_election.cc:304] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4d42b74842428593e2c2791df6e508, 6845fd5e9fbd42e88d452eb9865aab49; no voters:
I20260811 01:57:37.980000 13017 raft_consensus.cc:2804] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:37.980224 13017 raft_consensus.cc:697] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 1 LEADER]: Becoming Leader. State: Replica: 6845fd5e9fbd42e88d452eb9865aab49, State: Running, Role: LEADER
I20260811 01:57:37.980376 13017 consensus_queue.cc:237] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } }
I20260811 01:57:37.981885 12530 catalog_manager.cc:5719] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6845fd5e9fbd42e88d452eb9865aab49 (127.10.166.131). New cstate: current_term: 1 leader_uuid: "6845fd5e9fbd42e88d452eb9865aab49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6845fd5e9fbd42e88d452eb9865aab49" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 } health_report { overall_health: UNKNOWN } } }
W20260811 01:57:37.982527 12737 tablet.cc:2406] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:57:38.377992 13017 consensus_queue.cc:1048] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4d42b74842428593e2c2791df6e508" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:38.381233 13029 consensus_queue.cc:1048] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52fdc6bd434447699647a8ed1fd1cbe" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:39.027747 10906 tablet_replica.cc:333] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508: stopping tablet replica
I20260811 01:57:39.027920 10906 raft_consensus.cc:2243] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:39.028070 10906 raft_consensus.cc:2272] T ec6bda1e441c48fcbd5c3229769fa5cb P 1e4d42b74842428593e2c2791df6e508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:39.571208 12878 consensus_queue.cc:579] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [LEADER]: The tablet replica hosted on peer 1e4d42b74842428593e2c2791df6e508 has failed
W20260811 01:57:39.571362 12878 consensus_peers.cc:597] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 -> Peer 1e4d42b74842428593e2c2791df6e508 (127.10.166.129:39773): Couldn't send request to peer 1e4d42b74842428593e2c2791df6e508. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:40.055119 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:40.058539 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:40.070739 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:40.072551 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:40.075755 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:40.075932 10906 tablet_replica.cc:333] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe: stopping tablet replica
I20260811 01:57:40.076041 10906 raft_consensus.cc:2243] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:40.076141 10906 raft_consensus.cc:2272] T ec6bda1e441c48fcbd5c3229769fa5cb P c52fdc6bd434447699647a8ed1fd1cbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:40.088337 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:40.090013 10906 tablet_server.cc:179] TabletServer@127.10.166.131:0 shutting down...
I20260811 01:57:40.093298 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:40.093477 10906 tablet_replica.cc:333] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49: stopping tablet replica
I20260811 01:57:40.093580 10906 raft_consensus.cc:2243] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:40.093755 10906 raft_consensus.cc:2272] T ec6bda1e441c48fcbd5c3229769fa5cb P 6845fd5e9fbd42e88d452eb9865aab49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:40.105962 10906 tablet_server.cc:196] TabletServer@127.10.166.131:0 shutdown complete.
I20260811 01:57:40.107621 10906 master.cc:562] Master@127.10.166.190:33919 shutting down...
I20260811 01:57:40.110495 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:40.110622 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:40.110682 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 57fdade4dc9940b5af0ca6d152bd1b6e: stopping tablet replica
I20260811 01:57:40.122581 10906 master.cc:584] Master@127.10.166.190:33919 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (2475 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260811 01:57:40.128341 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:39851
I20260811 01:57:40.128662 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:40.130178 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.130209 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.130231 13053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.130358 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:40.130491 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.130527 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.130548 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460130546 us; error 0 us; skew 500 ppm
I20260811 01:57:40.131255 10906 webserver.cc:533] Webserver started at http://127.10.166.190:41005/ using document root <none> and password file <none>
I20260811 01:57:40.131421 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.131467 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.131536 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.131932 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "d5283ad6e7d443289cdeed4fd1fc7322"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.133330 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.134173 13061 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.134445 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.134511 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "d5283ad6e7d443289cdeed4fd1fc7322"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.134575 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.144280 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.144569 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.148370 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:39851
I20260811 01:57:40.150094 13140 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:39851 every 8 connection(s)
I20260811 01:57:40.150523 13141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.152221 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322: Bootstrap starting.
I20260811 01:57:40.152963 13141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.153832 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322: No bootstrap required, opened a new log
I20260811 01:57:40.154186 13141 raft_consensus.cc:359] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER }
I20260811 01:57:40.154263 13141 raft_consensus.cc:385] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.154289 13141 raft_consensus.cc:740] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5283ad6e7d443289cdeed4fd1fc7322, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.154397 13141 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER }
I20260811 01:57:40.154474 13141 raft_consensus.cc:399] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:40.154503 13141 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:40.154537 13141 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.155126 13141 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER }
I20260811 01:57:40.155234 13141 leader_election.cc:304] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5283ad6e7d443289cdeed4fd1fc7322; no voters:
I20260811 01:57:40.155378 13141 leader_election.cc:290] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:40.155494 13144 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:40.155676 13144 raft_consensus.cc:697] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 1 LEADER]: Becoming Leader. State: Replica: d5283ad6e7d443289cdeed4fd1fc7322, State: Running, Role: LEADER
I20260811 01:57:40.155831 13141 sys_catalog.cc:565] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:40.155831 13144 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER }
I20260811 01:57:40.156271 13146 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER } }
I20260811 01:57:40.156374 13146 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:40.156281 13147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5283ad6e7d443289cdeed4fd1fc7322. Latest consensus state: current_term: 1 leader_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5283ad6e7d443289cdeed4fd1fc7322" member_type: VOTER } }
I20260811 01:57:40.156482 13147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:40.156961 13153 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:40.157686 13153 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:40.157843 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:40.159368 13153 catalog_manager.cc:1383] Generated new cluster ID: 1c1bc1de13694e9f934d6ffaee8a6a70
I20260811 01:57:40.159417 13153 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:40.179726 13153 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:40.180207 13153 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:40.186631 13153 catalog_manager.cc:6092] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322: Generated new TSK 0
I20260811 01:57:40.186761 13153 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:40.189917 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:40.191597 13175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.191650 13176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.191746 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:40.191640 13179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.192008 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.192052 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.192066 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460192067 us; error 0 us; skew 500 ppm
I20260811 01:57:40.192813 10906 webserver.cc:533] Webserver started at http://127.10.166.129:35725/ using document root <none> and password file <none>
I20260811 01:57:40.192934 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.192971 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.193020 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.193313 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "90241e21ad354920beb9721e9265efa6"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.194607 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.195467 13187 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.195667 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.195749 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "90241e21ad354920beb9721e9265efa6"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.195818 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.200459 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.200731 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.200960 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:40.201350 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:40.201388 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.201437 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:40.201462 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.205224 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:36611
I20260811 01:57:40.205247 13285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:36611 every 8 connection(s)
I20260811 01:57:40.209635 13289 heartbeater.cc:344] Connected to a master server at 127.10.166.190:39851
I20260811 01:57:40.209717 13289 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.209873 13289 heartbeater.cc:507] Master 127.10.166.190:39851 requested a full tablet report, sending...
I20260811 01:57:40.210427 13091 ts_manager.cc:194] Registered new tserver with Master: 90241e21ad354920beb9721e9265efa6 (127.10.166.129:36611)
I20260811 01:57:40.211089 13091 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38370
I20260811 01:57:40.211098 10906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.005538264s
I20260811 01:57:40.216725 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:40.219448 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:40.231016 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:40.232648 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260811 01:57:40.233196 10906 tablet_server.cc:94] deduplicated master addresses: 127.10.166.190:39851
W20260811 01:57:40.234176 13309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.234238 13307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.234326 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:40.234440 13312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.234640 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.234683 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.234702 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460234702 us; error 0 us; skew 500 ppm
I20260811 01:57:40.235395 10906 webserver.cc:533] Webserver started at http://127.10.166.129:35725/ using document root <none> and password file <none>
I20260811 01:57:40.235556 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.235597 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:40.236543 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.237238 13321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.237443 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.237505 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "90241e21ad354920beb9721e9265efa6"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.237578 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.243567 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.243860 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.244099 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:40.244498 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:40.244535 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.244575 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:40.244603 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.248284 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:36611
I20260811 01:57:40.248330 13420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:36611 every 8 connection(s)
I20260811 01:57:40.249517 10906 tablet_server.cc:179] TabletServer@127.10.166.129:36611 shutting down...
I20260811 01:57:40.254117 13421 heartbeater.cc:344] Connected to a master server at 127.10.166.190:39851
I20260811 01:57:40.254207 13421 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.254388 13421 heartbeater.cc:507] Master 127.10.166.190:39851 requested a full tablet report, sending...
I20260811 01:57:40.254958 13091 ts_manager.cc:194] Re-registered known tserver with Master: 90241e21ad354920beb9721e9265efa6 (127.10.166.129:36611)
I20260811 01:57:40.255450 13091 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38398
I20260811 01:57:40.256137 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:40.267855 10906 tablet_server.cc:196] TabletServer@127.10.166.129:36611 shutdown complete.
I20260811 01:57:40.269487 10906 master.cc:562] Master@127.10.166.190:39851 shutting down...
I20260811 01:57:40.272770 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:40.272915 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:40.273020 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P d5283ad6e7d443289cdeed4fd1fc7322: stopping tablet replica
I20260811 01:57:40.285175 10906 master.cc:584] Master@127.10.166.190:39851 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (161 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260811 01:57:40.289961 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:32923,127.10.166.189:38757,127.10.166.188:36879
I20260811 01:57:40.290344 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:40.291805 13435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.291826 13439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.291901 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:40.291815 13434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.292110 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.292153 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.292167 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460292167 us; error 0 us; skew 500 ppm
I20260811 01:57:40.292832 10906 webserver.cc:533] Webserver started at http://127.10.166.190:40255/ using document root <none> and password file <none>
I20260811 01:57:40.292964 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.293004 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.293071 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.293406 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "ba2ce60fed75403dbc1420a12e875e04"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.294739 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:40.295560 13444 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.295783 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.295848 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "ba2ce60fed75403dbc1420a12e875e04"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.295914 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.300570 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.300832 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.304445 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:32923
I20260811 01:57:40.305454 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260811 01:57:40.305437 13532 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:32923 every 8 connection(s)
I20260811 01:57:40.305850 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 01:57:40.306821 13535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.306840 13539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.306960 13537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.307175 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:40.307226 13533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.307395 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.307439 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.307461 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460307460 us; error 0 us; skew 500 ppm
I20260811 01:57:40.308243 10906 webserver.cc:533] Webserver started at http://127.10.166.189:40561/ using document root <none> and password file <none>
I20260811 01:57:40.308379 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.308434 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.308498 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.308840 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root/instance:
uuid: "9deebe6f8825456ba2e6b67845bb95fc"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.310592 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 01:57:40.311558 13554 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.311650 13533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.311836 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.311906 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
uuid: "9deebe6f8825456ba2e6b67845bb95fc"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.311959 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 01:57:40.312403 13449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.166.189:38757: connect: Connection refused (error 111)
W20260811 01:57:40.313151 13533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.166.189:38757: Network error: Client connection negotiation failed: client connection to 127.10.166.189:38757: connect: Connection refused (error 111)
I20260811 01:57:40.330238 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.330546 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.334270 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.189:38757
I20260811 01:57:40.334295 13632 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.189:38757 every 8 connection(s)
I20260811 01:57:40.334959 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260811 01:57:40.335068 13633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 01:57:40.336411 13636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.336467 13635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.336639 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:40.336658 13633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:57:40.336501 13641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.336959 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.337009 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.337024 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460337024 us; error 0 us; skew 500 ppm
I20260811 01:57:40.337757 10906 webserver.cc:533] Webserver started at http://127.10.166.188:43357/ using document root <none> and password file <none>
I20260811 01:57:40.337918 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.337972 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.338032 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.338445 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root/instance:
uuid: "6a4c4821171c424fb6bcb1b00ebf137c"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.340205 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:40.340997 13633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.341290 13649 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.341603 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.341668 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
uuid: "6a4c4821171c424fb6bcb1b00ebf137c"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.341733 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.344837 13633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:57:40.345996 13633 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.166.188:36879: Network error: Client connection negotiation failed: client connection to 127.10.166.188:36879: connect: Connection refused (error 111)
I20260811 01:57:40.357311 13533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } attempt: 1
I20260811 01:57:40.360865 13533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:57:40.361977 13533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.10.166.188:36879: Network error: Client connection negotiation failed: client connection to 127.10.166.188:36879: connect: Connection refused (error 111)
I20260811 01:57:40.363044 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.363381 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.367799 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.188:36879
I20260811 01:57:40.367846 13735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.188:36879 every 8 connection(s)
I20260811 01:57:40.368216 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:40.368614 13736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.369633 13736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.372931 13736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.375875 13736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:57:40.379125 13533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } attempt: 1
I20260811 01:57:40.379132 13633 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } attempt: 1
I20260811 01:57:40.380470 13736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Bootstrap starting.
I20260811 01:57:40.381494 13736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.382910 13736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: No bootstrap required, opened a new log
I20260811 01:57:40.383494 13736 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.383596 13736 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.383628 13736 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4c4821171c424fb6bcb1b00ebf137c, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.383839 13736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.384121 13533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Bootstrap starting.
I20260811 01:57:40.384315 13633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Bootstrap starting.
I20260811 01:57:40.384337 13736 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:40.385176 13533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.385231 13633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.385919 13745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.386287 13533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: No bootstrap required, opened a new log
I20260811 01:57:40.386400 13745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.386672 13633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: No bootstrap required, opened a new log
W20260811 01:57:40.386848 13763 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:57:40.386893 13763 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:57:40.386821 13533 raft_consensus.cc:359] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.386989 13533 raft_consensus.cc:385] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.387046 13533 raft_consensus.cc:740] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2ce60fed75403dbc1420a12e875e04, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.387025 13633 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.387115 13633 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.387151 13633 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9deebe6f8825456ba2e6b67845bb95fc, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.387171 13533 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.387250 13633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.387473 13764 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.387598 13764 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.387501 13766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.387650 13533 sys_catalog.cc:565] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:40.387650 13633 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:40.387681 13766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.389364 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260811 01:57:40.389418 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20260811 01:57:40.389799 13796 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:57:40.389853 13796 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:57:40.389833 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260811 01:57:40.390185 10906 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260811 01:57:40.391492 13797 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:57:40.391518 13799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.391532 13801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.391719 13797 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:57:40.391645 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:40.391942 13803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.392213 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.392256 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.392277 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460392276 us; error 0 us; skew 500 ppm
I20260811 01:57:40.392982 10906 webserver.cc:533] Webserver started at http://127.10.166.129:43867/ using document root <none> and password file <none>
I20260811 01:57:40.393121 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.393167 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.393236 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.393567 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "d04f1bc272f941f696a322d68fa2dddf"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.394862 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:40.395679 13811 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.395941 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.396004 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "d04f1bc272f941f696a322d68fa2dddf"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.396064 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.414707 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.415060 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.415339 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:40.415844 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:40.415891 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.415935 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:40.415966 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.422300 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:42005
I20260811 01:57:40.422322 13909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:42005 every 8 connection(s)
I20260811 01:57:40.423249 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260811 01:57:40.423750 10906 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260811 01:57:40.425439 13924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.426501 13920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.426587 13921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.426607 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:40.427063 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.427117 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.427138 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460427137 us; error 0 us; skew 500 ppm
I20260811 01:57:40.428018 10906 webserver.cc:533] Webserver started at http://127.10.166.130:42791/ using document root <none> and password file <none>
I20260811 01:57:40.428027 13911 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:40.428157 13911 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.428191 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.428241 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.428303 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.428381 13911 heartbeater.cc:507] Master 127.10.166.190:32923 requested a full tablet report, sending...
I20260811 01:57:40.428630 13910 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:40.428710 13910 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.428755 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.428889 13910 heartbeater.cc:507] Master 127.10.166.188:36879 requested a full tablet report, sending...
I20260811 01:57:40.429033 13470 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.429535 13682 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.429675 13912 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:40.429744 13912 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.429946 13912 heartbeater.cc:507] Master 127.10.166.189:38757 requested a full tablet report, sending...
I20260811 01:57:40.430555 13577 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.430563 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:40.431552 13936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.431806 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.431871 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.431941 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.447053 13764 raft_consensus.cc:493] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:40.447185 13764 raft_consensus.cc:515] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.447649 13764 leader_election.cc:290] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
I20260811 01:57:40.448058 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc" is_pre_election: true
I20260811 01:57:40.448061 13698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" is_pre_election: true
I20260811 01:57:40.448200 13596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2ce60fed75403dbc1420a12e875e04 in term 0.
I20260811 01:57:40.448233 13698 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2ce60fed75403dbc1420a12e875e04 in term 0.
I20260811 01:57:40.448624 13449 leader_election.cc:304] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9deebe6f8825456ba2e6b67845bb95fc, ba2ce60fed75403dbc1420a12e875e04; no voters:
I20260811 01:57:40.448801 13764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:40.448863 13764 raft_consensus.cc:493] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:40.448896 13764 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.449772 13764 raft_consensus.cc:515] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.450114 13764 leader_election.cc:290] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 1 election: Requested vote from peers 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
I20260811 01:57:40.450388 13698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4c4821171c424fb6bcb1b00ebf137c"
I20260811 01:57:40.450402 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc"
I20260811 01:57:40.450501 13698 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.450525 13596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.451350 13596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2ce60fed75403dbc1420a12e875e04 in term 1.
I20260811 01:57:40.451507 13698 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2ce60fed75403dbc1420a12e875e04 in term 1.
I20260811 01:57:40.451597 13449 leader_election.cc:304] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9deebe6f8825456ba2e6b67845bb95fc, ba2ce60fed75403dbc1420a12e875e04; no voters:
I20260811 01:57:40.451761 13764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:40.451992 13764 raft_consensus.cc:697] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 LEADER]: Becoming Leader. State: Replica: ba2ce60fed75403dbc1420a12e875e04, State: Running, Role: LEADER
I20260811 01:57:40.452165 13764 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:40.452801 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba2ce60fed75403dbc1420a12e875e04. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.452899 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:40.453204 13948 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:40.453898 13948 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:40.455603 13596 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Refusing update from remote peer ba2ce60fed75403dbc1420a12e875e04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.455603 13698 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Refusing update from remote peer ba2ce60fed75403dbc1420a12e875e04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.456003 13945 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.456127 13764 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.457641 13745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba2ce60fed75403dbc1420a12e875e04. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.457736 13745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.457809 13766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba2ce60fed75403dbc1420a12e875e04. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.457870 13766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.458702 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.459111 13764 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.459152 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.459152 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.459276 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:40.459227 13764 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:40.459296 13949 mvcc.cc:204] Tried to move back new op lower bound from 7317149534022995968 to 7317149534013480960. Current Snapshot: MvccSnapshot[applied={T|T < 7317149534022995968}]
I20260811 01:57:40.459515 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:40.459753 13948 catalog_manager.cc:1383] Generated new cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:40.459795 13948 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:40.459842 13745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.459925 13745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.460083 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:40.460121 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.460160 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:40.460248 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.460160 13766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba2ce60fed75403dbc1420a12e875e04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:40.460289 13766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:40.466084 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:39917
I20260811 01:57:40.466159 14068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:39917 every 8 connection(s)
I20260811 01:57:40.467046 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260811 01:57:40.467449 10906 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
I20260811 01:57:40.472534 14069 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:40.472623 14069 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.472679 13948 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:40.472824 14069 heartbeater.cc:507] Master 127.10.166.188:36879 requested a full tablet report, sending...
I20260811 01:57:40.473079 14072 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:40.473196 14072 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.473330 13948 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:40.473479 14072 heartbeater.cc:507] Master 127.10.166.190:32923 requested a full tablet report, sending...
I20260811 01:57:40.473603 13682 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.474015 13470 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
W20260811 01:57:40.475224 14084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.475775 14083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:40.475914 14086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:40.476078 14075 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:40.476152 14075 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.476235 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:40.476361 14075 heartbeater.cc:507] Master 127.10.166.189:38757 requested a full tablet report, sending...
I20260811 01:57:40.476442 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:40.476485 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:40.476542 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413460476541 us; error 0 us; skew 500 ppm
I20260811 01:57:40.476984 13577 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.477488 10906 webserver.cc:533] Webserver started at http://127.10.166.131:41087/ using document root <none> and password file <none>
I20260811 01:57:40.477639 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:40.477691 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:40.477770 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:40.478159 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-2-root/instance:
uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.479765 13948 catalog_manager.cc:6092] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Generated new TSK 0
I20260811 01:57:40.479898 13948 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:40.480959 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:57:40.481984 14096 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.482199 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.482256 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-2-root
uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:40.482318 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:40.492439 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:40.492772 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:40.493036 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:40.493479 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:40.493516 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.493551 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:40.493578 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:40.498412 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.131:34463
I20260811 01:57:40.498514 14198 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.131:34463 every 8 connection(s)
I20260811 01:57:40.503608 14200 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:40.503633 14202 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:40.503727 14200 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.503736 14202 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.503822 14201 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:40.503880 14201 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:40.503940 14200 heartbeater.cc:507] Master 127.10.166.188:36879 requested a full tablet report, sending...
I20260811 01:57:40.503973 14202 heartbeater.cc:507] Master 127.10.166.189:38757 requested a full tablet report, sending...
I20260811 01:57:40.504022 14201 heartbeater.cc:507] Master 127.10.166.190:32923 requested a full tablet report, sending...
I20260811 01:57:40.504514 13470 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:40.504577 13577 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:40.504577 13682 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:40.504750 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005808155s
I20260811 01:57:40.505425 13470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50384
I20260811 01:57:40.512753 13470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260811 01:57:40.513171 13470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:57:40.525550 14137 tablet_service.cc:1511] Processing CreateTablet for tablet 2310d0060d5e49c1b73c635b530d1d2f (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.525697 14136 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9b8e0a66174c639a28dead93e92e36 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.525841 14137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310d0060d5e49c1b73c635b530d1d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.525986 14136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9b8e0a66174c639a28dead93e92e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.526578 14002 tablet_service.cc:1511] Processing CreateTablet for tablet 2310d0060d5e49c1b73c635b530d1d2f (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.526813 14002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310d0060d5e49c1b73c635b530d1d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.528143 13859 tablet_service.cc:1511] Processing CreateTablet for tablet 2310d0060d5e49c1b73c635b530d1d2f (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.528390 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310d0060d5e49c1b73c635b530d1d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.528425 14135 tablet_service.cc:1511] Processing CreateTablet for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.528671 14135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.528885 14232 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:40.529043 14001 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9b8e0a66174c639a28dead93e92e36 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.529268 14001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9b8e0a66174c639a28dead93e92e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.530228 14232 tablet_bootstrap.cc:654] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.530493 13856 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9b8e0a66174c639a28dead93e92e36 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.530747 13856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9b8e0a66174c639a28dead93e92e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.531275 14000 tablet_service.cc:1511] Processing CreateTablet for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.531503 14000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.532133 13855 tablet_service.cc:1511] Processing CreateTablet for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (DEFAULT_TABLE table=test-table [id=3c940178366b45f281a1f6e9f43c2ecd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:40.532604 13855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:40.532783 14233 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:40.533762 14233 tablet_bootstrap.cc:654] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.534070 14231 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: Bootstrap starting.
I20260811 01:57:40.534399 13470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50356
I20260811 01:57:40.534824 14232 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: No bootstrap required, opened a new log
I20260811 01:57:40.534933 14232 ts_tablet_manager.cc:1403] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 01:57:40.535418 14232 raft_consensus.cc:359] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.535533 14232 raft_consensus.cc:385] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.535567 14232 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.535763 14232 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.536161 14232 ts_tablet_manager.cc:1434] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:40.536289 14233 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: No bootstrap required, opened a new log
I20260811 01:57:40.536346 14232 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:40.536355 14233 ts_tablet_manager.cc:1403] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:57:40.536828 14233 raft_consensus.cc:359] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.536921 14233 raft_consensus.cc:385] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.536950 14233 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.537065 14233 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.537307 14232 tablet_bootstrap.cc:654] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.537390 14233 ts_tablet_manager.cc:1434] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.537626 13469 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50368
I20260811 01:57:40.539492 14072 heartbeater.cc:499] Master 127.10.166.190:32923 was elected leader, sending a full tablet report...
I20260811 01:57:40.539944 14232 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: No bootstrap required, opened a new log
I20260811 01:57:40.540091 14232 ts_tablet_manager.cc:1403] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:57:40.540117 13911 heartbeater.cc:499] Master 127.10.166.190:32923 was elected leader, sending a full tablet report...
I20260811 01:57:40.540272 14233 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:40.541298 14233 tablet_bootstrap.cc:654] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.541462 14232 raft_consensus.cc:359] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.541551 14232 raft_consensus.cc:385] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.541584 14232 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.541698 14232 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.541980 14232 ts_tablet_manager.cc:1434] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:40.543452 14232 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:40.535561 14231 tablet_bootstrap.cc:654] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.545302 14233 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: No bootstrap required, opened a new log
I20260811 01:57:40.545367 14232 tablet_bootstrap.cc:654] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.545373 14233 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 01:57:40.545918 14233 raft_consensus.cc:359] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.546029 14233 raft_consensus.cc:385] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.546064 14233 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.546177 14233 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.546336 14231 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: No bootstrap required, opened a new log
I20260811 01:57:40.546398 14233 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.546416 14231 ts_tablet_manager.cc:1403] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260811 01:57:40.546566 14233 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:40.546872 14231 raft_consensus.cc:359] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.546965 14231 raft_consensus.cc:385] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.546994 14231 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.547413 14232 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: No bootstrap required, opened a new log
I20260811 01:57:40.547478 14233 tablet_bootstrap.cc:654] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.547487 14232 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 01:57:40.547919 14232 raft_consensus.cc:359] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.547998 14232 raft_consensus.cc:385] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.548017 14232 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.548125 14232 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.548364 14232 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.548419 14237 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:40.548601 14237 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.548663 14233 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: No bootstrap required, opened a new log
I20260811 01:57:40.548735 14233 ts_tablet_manager.cc:1403] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:40.549129 14237 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.551800 14231 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.552117 14249 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:40.552171 14231 ts_tablet_manager.cc:1434] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260811 01:57:40.552248 14249 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.553363 14201 heartbeater.cc:499] Master 127.10.166.190:32923 was elected leader, sending a full tablet report...
I20260811 01:57:40.553555 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:40.553656 14019 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 0.
I20260811 01:57:40.553642 14231 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: Bootstrap starting.
I20260811 01:57:40.554041 13812 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:40.555413 14231 tablet_bootstrap.cc:654] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.556594 14237 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:40.556674 14237 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:40.556723 14237 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.556669 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
W20260811 01:57:40.557133 13813 leader_election.cc:343] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463): Illegal state: must be running to vote when last-logged opid is not known
I20260811 01:57:40.557731 14237 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.557976 14231 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: No bootstrap required, opened a new log
I20260811 01:57:40.558002 14249 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.558053 14231 ts_tablet_manager.cc:1403] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:57:40.558135 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:40.558230 14019 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 0.
I20260811 01:57:40.558555 14231 raft_consensus.cc:359] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.558645 14231 raft_consensus.cc:385] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.558063 14237 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 election: Requested vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.558691 14231 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.558853 14231 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.559123 14231 ts_tablet_manager.cc:1434] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.559208 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:40.559302 14019 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.559309 14231 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Bootstrap starting.
I20260811 01:57:40.560293 14019 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 1.
I20260811 01:57:40.560468 14231 tablet_bootstrap.cc:654] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:40.560855 14097 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:40.549135 14233 raft_consensus.cc:359] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.561064 14233 raft_consensus.cc:385] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.561079 14248 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:40.561117 14233 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.561172 14248 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:40.561223 14248 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.561261 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:40.561244 14233 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.561336 14156 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.561486 14233 ts_tablet_manager.cc:1434] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20260811 01:57:40.562090 14248 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.562263 14156 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 1.
I20260811 01:57:40.562597 14248 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 1 election: Requested vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:40.563555 13812 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:40.563898 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:40.563954 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:40.564021 13875 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 0.
I20260811 01:57:40.564730 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:40.564814 14019 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.565142 14237 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:40.565359 14237 raft_consensus.cc:697] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Becoming Leader. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Running, Role: LEADER
I20260811 01:57:40.565526 14019 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 1.
I20260811 01:57:40.565536 14237 consensus_queue.cc:237] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.565889 14097 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:40.566056 14248 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:40.566134 14248 raft_consensus.cc:697] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 LEADER]: Becoming Leader. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:40.566241 14248 consensus_queue.cc:237] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.567853 13470 catalog_manager.cc:5719] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf reported cstate change: term changed from 0 to 1, leader changed from <none> to d04f1bc272f941f696a322d68fa2dddf (127.10.166.129). New cstate: current_term: 1 leader_uuid: "d04f1bc272f941f696a322d68fa2dddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:40.570819 14237 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:40.570899 14237 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.571228 14237 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:40.571858 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:40.571969 14019 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 0.
I20260811 01:57:40.572081 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:40.572278 13812 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:40.572255 13470 catalog_manager.cc:5719] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131). New cstate: current_term: 1 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } }
W20260811 01:57:40.572351 13813 leader_election.cc:343] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463): Illegal state: must be running to vote when last-logged opid is not known
I20260811 01:57:40.572646 14237 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:40.572697 14237 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:40.572738 14237 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.588218 14231 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: No bootstrap required, opened a new log
I20260811 01:57:40.588313 14231 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20260811 01:57:40.588405 14237 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.588769 14237 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 election: Requested vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:40.588812 14231 raft_consensus.cc:359] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.588972 14231 raft_consensus.cc:385] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:40.589048 14019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:40.589056 14231 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Initialized, Role: FOLLOWER
I20260811 01:57:40.589069 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:40.589195 14019 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.589314 14231 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.589480 14156 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.589666 14231 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:40.590116 14019 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 1.
I20260811 01:57:40.590315 14156 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 1.
I20260811 01:57:40.590391 13812 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:40.590552 14237 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:40.590637 14237 raft_consensus.cc:697] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Becoming Leader. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Running, Role: LEADER
I20260811 01:57:40.590754 14237 consensus_queue.cc:237] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.593182 13469 catalog_manager.cc:5719] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf reported cstate change: term changed from 0 to 1, leader changed from <none> to d04f1bc272f941f696a322d68fa2dddf (127.10.166.129). New cstate: current_term: 1 leader_uuid: "d04f1bc272f941f696a322d68fa2dddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:40.637781 14019 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.638018 14156 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.638072 14155 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.638240 14253 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.638423 14237 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.638468 14019 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.638521 14253 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.641852 14267 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.642316 14017 raft_consensus.cc:1275] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.642885 14249 consensus_queue.cc:1048] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.643858 13873 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:40.645004 13873 raft_consensus.cc:1275] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:57:40.645542 14248 consensus_queue.cc:1048] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:40.677414 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f"
dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
from {username='slave'} at 127.0.0.1:38408
I20260811 01:57:40.677610 13873 raft_consensus.cc:591] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Received request to step down
I20260811 01:57:40.677668 13873 raft_consensus.cc:3055] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:40.677719 13873 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Running, Role: LEADER
I20260811 01:57:40.677855 13873 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.678006 13873 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:40.846971 14248 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:40.847117 14248 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.847456 14248 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.847785 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:40.847813 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:40.847920 13873 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 2.
I20260811 01:57:40.848189 14100 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:40.848345 14248 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:40.848402 14248 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting leader election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:40.848457 14248 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:40.849390 14248 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.849706 14248 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 election: Requested vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.849983 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:40.849983 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:40.850827 13873 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 2.
I20260811 01:57:40.851065 14100 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:40.851222 14248 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:40.851322 14248 raft_consensus.cc:697] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 LEADER]: Becoming Leader. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:40.851480 14248 consensus_queue.cc:237] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.853221 14236 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:40.853296 14236 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:40.853315 13469 catalog_manager.cc:5719] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: term changed from 1 to 2, leader changed from d04f1bc272f941f696a322d68fa2dddf (127.10.166.129) to 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131). New cstate: current_term: 2 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:40.853731 14236 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:40.857996 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:40.859441 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:40.859591 13873 raft_consensus.cc:2393] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc7d76b1ea514ec18d0cc66531ce6d0a in current term 2: Already voted for candidate 8f97d94fcabc42b89b54eb018bbf9fde in this term.
I20260811 01:57:40.859864 13941 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:40.860033 14236 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:40.860962 14236 raft_consensus.cc:2749] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:57:40.879827 13873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0"
dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
from {username='slave'} at 127.0.0.1:38408
I20260811 01:57:40.879949 13873 raft_consensus.cc:591] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Received request to step down
I20260811 01:57:40.879997 13873 raft_consensus.cc:3055] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:40.880023 13873 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Running, Role: LEADER
I20260811 01:57:40.880151 13873 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:40.880311 13873 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:40.934185 13873 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 01:57:40.934620 14248 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:40.937011 14016 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 01:57:40.937566 14249 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:41.047858 14297 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:41.048034 14297 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.048466 14297 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:41.048810 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:41.048816 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:41.048959 13873 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 2.
I20260811 01:57:41.049201 13941 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde
I20260811 01:57:41.049418 14297 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:41.049496 14297 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:41.049544 14297 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.050520 14297 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.050836 14297 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 election: Requested vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:41.051155 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:41.051155 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:41.052659 13873 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 2.
I20260811 01:57:41.052969 13941 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde
I20260811 01:57:41.053197 14297 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:41.053565 14297 raft_consensus.cc:697] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Becoming Leader. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Running, Role: LEADER
I20260811 01:57:41.053723 14297 consensus_queue.cc:237] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.055003 13469 catalog_manager.cc:5719] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a reported cstate change: term changed from 1 to 2, leader changed from d04f1bc272f941f696a322d68fa2dddf (127.10.166.129) to fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130). New cstate: current_term: 2 leader_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:41.055824 14248 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:41.055915 14248 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.056536 14248 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:41.057037 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:41.057193 13873 raft_consensus.cc:2393] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f97d94fcabc42b89b54eb018bbf9fde in current term 2: Already voted for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in this term.
I20260811 01:57:41.057592 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:41.057928 14097 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde; no voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.058333 14248 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.059399 14248 raft_consensus.cc:2749] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:57:41.086504 14016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0"
dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
from {username='slave'} at 127.0.0.1:60572
I20260811 01:57:41.086625 14016 raft_consensus.cc:591] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Received request to step down
I20260811 01:57:41.086674 14016 raft_consensus.cc:3055] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:41.086707 14016 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Running, Role: LEADER
I20260811 01:57:41.086872 14016 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.086990 14016 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.292263 14156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36"
dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
from {username='slave'} at 127.0.0.1:39728
I20260811 01:57:41.292382 14156 raft_consensus.cc:591] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 LEADER]: Received request to step down
I20260811 01:57:41.292418 14156 raft_consensus.cc:3055] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 LEADER]: Stepping down as leader of term 1
I20260811 01:57:41.292449 14156 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:41.292582 14156 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.292713 14156 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.345934 14248 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:41.346056 14248 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.346462 14248 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:41.346823 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:41.346823 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:41.346989 13873 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 2.
I20260811 01:57:41.347025 14016 raft_consensus.cc:2410] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f97d94fcabc42b89b54eb018bbf9fde for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260811 01:57:41.347290 14100 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.347483 14248 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:57:41.347558 14248 raft_consensus.cc:493] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:41.347589 14248 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.348451 14248 raft_consensus.cc:515] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.348766 14248 leader_election.cc:290] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 election: Requested vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:41.349049 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:41.349049 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9a2857d50d48d3bdfcc7c6e0d7fcf0" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:41.349181 13873 raft_consensus.cc:3060] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.349217 14016 raft_consensus.cc:2410] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f97d94fcabc42b89b54eb018bbf9fde for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260811 01:57:41.350041 13873 raft_consensus.cc:2468] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 3.
I20260811 01:57:41.350281 14100 leader_election.cc:304] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.350436 14248 raft_consensus.cc:2804] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:57:41.350525 14248 raft_consensus.cc:697] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Becoming Leader. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:41.350657 14248 consensus_queue.cc:237] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.352263 13466 catalog_manager.cc:5719] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: term changed from 2 to 3, leader changed from fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) to 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131). New cstate: current_term: 3 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:41.387885 13763 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:41.387959 13763 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: loading cluster ID for follower catalog manager: success
I20260811 01:57:41.389693 13763 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: acquiring CA information for follower catalog manager: success
I20260811 01:57:41.390647 13763 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:57:41.392499 13797 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:41.392545 13797 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: loading cluster ID for follower catalog manager: success
I20260811 01:57:41.393558 13797 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: acquiring CA information for follower catalog manager: success
I20260811 01:57:41.394052 13797 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:57:41.427834 13873 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:41.428254 14248 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:41.431058 14016 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260811 01:57:41.431130 14016 pending_rounds.cc:85] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Aborting all ops after (but not including) 2
I20260811 01:57:41.431183 14016 pending_rounds.cc:107] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260811 01:57:41.431216 14016 raft_consensus.cc:2889] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260811 01:57:41.432493 14248 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:41.432668 14297 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.432756 14297 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.433130 14297 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:41.433380 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:41.433490 14156 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 2.
I20260811 01:57:41.433334 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:41.433981 13939 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:41.434257 14297 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:41.434314 14297 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.434355 14297 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.435532 14297 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.435921 14297 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 election: Requested vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:41.436187 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:41.436197 14156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:41.436540 14267 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.436620 14267 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.436937 14267 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:41.437099 14156 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 2.
I20260811 01:57:41.437186 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:41.437237 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:41.437304 14016 raft_consensus.cc:2393] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d04f1bc272f941f696a322d68fa2dddf in current term 2: Already voted for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in this term.
I20260811 01:57:41.437357 14155 raft_consensus.cc:2393] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d04f1bc272f941f696a322d68fa2dddf in current term 2: Already voted for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in this term.
I20260811 01:57:41.437337 13939 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:41.437566 14297 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:41.437595 13812 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf; no voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.437660 14297 raft_consensus.cc:697] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Becoming Leader. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Running, Role: LEADER
I20260811 01:57:41.437793 14297 consensus_queue.cc:237] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:41.437970 14267 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.438839 14267 raft_consensus.cc:2749] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:57:41.439127 13466 catalog_manager.cc:5719] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a reported cstate change: term changed from 1 to 2, leader changed from 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131) to fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130). New cstate: current_term: 2 leader_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:41.494797 14155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f"
dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
from {username='slave'} at 127.0.0.1:39728
I20260811 01:57:41.494938 14155 raft_consensus.cc:591] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 LEADER]: Received request to step down
I20260811 01:57:41.494988 14155 raft_consensus.cc:3055] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 LEADER]: Stepping down as leader of term 2
I20260811 01:57:41.495015 14155 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:41.495146 14155 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:41.495306 14155 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.544703 14155 raft_consensus.cc:1275] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Refusing update from remote peer fc7d76b1ea514ec18d0cc66531ce6d0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 01:57:41.545075 14297 consensus_queue.cc:1048] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:41.547295 13873 raft_consensus.cc:1275] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Refusing update from remote peer fc7d76b1ea514ec18d0cc66531ce6d0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 01:57:41.548434 14297 consensus_queue.cc:1048] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:41.671283 14267 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.671417 14267 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:41.671792 14267 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:41.672114 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:41.672120 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:41.672273 14155 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 3.
I20260811 01:57:41.672530 13813 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.672683 14267 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:57:41.672740 14267 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.672791 14267 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.673668 14267 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:41.674001 14267 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 election: Requested vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:41.674250 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:41.674279 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:41.675169 14155 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d04f1bc272f941f696a322d68fa2dddf in term 3.
I20260811 01:57:41.675395 13813 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf; no voters: fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:41.675545 14267 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:57:41.675740 14267 raft_consensus.cc:697] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 LEADER]: Becoming Leader. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Running, Role: LEADER
I20260811 01:57:41.675897 14267 consensus_queue.cc:237] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:41.677326 13466 catalog_manager.cc:5719] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf reported cstate change: term changed from 2 to 3, leader changed from 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131) to d04f1bc272f941f696a322d68fa2dddf (127.10.166.129). New cstate: current_term: 3 leader_uuid: "d04f1bc272f941f696a322d68fa2dddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:41.696981 14303 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f97d94fcabc42b89b54eb018bbf9fde)
I20260811 01:57:41.697101 13469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50396:
table { table_id: "3c940178366b45f281a1f6e9f43c2ecd" table_name: "test-table" } new_table_name: "NewTableName"
I20260811 01:57:41.697103 14303 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:41.697731 14303 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:41.698186 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:41.698312 14155 raft_consensus.cc:2393] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc7d76b1ea514ec18d0cc66531ce6d0a in current term 3: Already voted for candidate d04f1bc272f941f696a322d68fa2dddf in this term.
I20260811 01:57:41.700132 13873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:41.700428 13941 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde, d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:41.700685 14303 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:41.701817 14303 raft_consensus.cc:2749] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260811 01:57:41.702962 10906 master.cc:562] Master@127.10.166.190:32923 shutting down...
I20260811 01:57:41.703469 14016 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 01:57:41.703943 14156 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Refusing update from remote peer d04f1bc272f941f696a322d68fa2dddf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 01:57:41.704613 14267 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:41.705513 14267 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:41.708246 14309 tablet.cc:1724] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.708956 14307 tablet.cc:1724] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.710449 14305 tablet.cc:1724] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.710777 14312 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.712695 14310 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.713044 14314 tablet.cc:1724] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.713301 14309 tablet.cc:1724] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.714749 14269 mvcc.cc:204] Tried to move back new op lower bound from 7317149539133657088 to 7317149539025887232. Current Snapshot: MvccSnapshot[applied={T|T < 7317149539133657088}]
I20260811 01:57:41.714830 14305 tablet.cc:1724] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.719653 14309 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:41.722344 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:41.722517 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:41.722579 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: stopping tablet replica
I20260811 01:57:41.735391 10906 master.cc:584] Master@127.10.166.190:32923 shutdown complete.
I20260811 01:57:41.901605 14324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba2ce60fed75403dbc1420a12e875e04)
I20260811 01:57:41.901752 14324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.901858 14325 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba2ce60fed75403dbc1420a12e875e04)
I20260811 01:57:41.901983 14325 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.902256 14324 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
I20260811 01:57:41.902910 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc" is_pre_election: true
I20260811 01:57:41.903040 13596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 1.
I20260811 01:57:41.903337 13656 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc; no voters:
I20260811 01:57:41.903375 13698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9deebe6f8825456ba2e6b67845bb95fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" is_pre_election: true
I20260811 01:57:41.903484 13698 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9deebe6f8825456ba2e6b67845bb95fc in term 1.
I20260811 01:57:41.903563 14324 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:41.903606 14324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba2ce60fed75403dbc1420a12e875e04)
I20260811 01:57:41.903636 14325 leader_election.cc:290] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
I20260811 01:57:41.903643 14324 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.903880 13561 leader_election.cc:304] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc; no voters:
W20260811 01:57:41.903980 13652 leader_election.cc:336] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923): Network error: Client connection negotiation failed: client connection to 127.10.166.190:32923: connect: Connection refused (error 111)
I20260811 01:57:41.904043 14325 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:41.904083 14325 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba2ce60fed75403dbc1420a12e875e04)
W20260811 01:57:41.904119 13556 leader_election.cc:336] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923): Network error: Client connection negotiation failed: client connection to 127.10.166.190:32923: connect: Connection refused (error 111)
I20260811 01:57:41.904116 14325 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:41.905047 14324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.905176 14325 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.905414 14324 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 election: Requested vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
I20260811 01:57:41.905783 13698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9deebe6f8825456ba2e6b67845bb95fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6a4c4821171c424fb6bcb1b00ebf137c"
I20260811 01:57:41.905900 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc"
I20260811 01:57:41.905910 13698 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9deebe6f8825456ba2e6b67845bb95fc in current term 2: Already voted for candidate 6a4c4821171c424fb6bcb1b00ebf137c in this term.
I20260811 01:57:41.906014 13596 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a4c4821171c424fb6bcb1b00ebf137c in current term 2: Already voted for candidate 9deebe6f8825456ba2e6b67845bb95fc in this term.
I20260811 01:57:41.906167 14325 leader_election.cc:290] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 election: Requested vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
W20260811 01:57:41.906325 13652 leader_election.cc:336] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923): Network error: Client connection negotiation failed: client connection to 127.10.166.190:32923: connect: Connection refused (error 111)
I20260811 01:57:41.906620 13656 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c; no voters: 9deebe6f8825456ba2e6b67845bb95fc, ba2ce60fed75403dbc1420a12e875e04
W20260811 01:57:41.906711 13556 leader_election.cc:336] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923): Network error: Client connection negotiation failed: client connection to 127.10.166.190:32923: connect: Connection refused (error 111)
I20260811 01:57:41.906775 14324 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 01:57:41.906785 13556 leader_election.cc:304] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9deebe6f8825456ba2e6b67845bb95fc; no voters: 6a4c4821171c424fb6bcb1b00ebf137c, ba2ce60fed75403dbc1420a12e875e04
I20260811 01:57:41.906948 14325 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 01:57:41.937914 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:41.939342 14335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:41.939407 14333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:41.939461 14337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:41.939591 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:41.939772 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:41.939812 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:41.939826 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413461939826 us; error 0 us; skew 500 ppm
I20260811 01:57:41.940747 10906 webserver.cc:533] Webserver started at http://127.10.166.190:40255/ using document root <none> and password file <none>
I20260811 01:57:41.940907 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:41.940953 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:41.941921 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:41.942742 14344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:41.942984 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:41.943051 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "ba2ce60fed75403dbc1420a12e875e04"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:41.943111 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:41.953600 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:41.953847 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:41.957535 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:32923
I20260811 01:57:41.958144 14426 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:32923 every 8 connection(s)
I20260811 01:57:41.959584 14427 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:57:41.960336 14427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Bootstrap starting.
I20260811 01:57:41.970752 14427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=7 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:41.971119 14427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: Bootstrap complete.
I20260811 01:57:41.971627 14427 raft_consensus.cc:359] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.971743 14427 raft_consensus.cc:740] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2ce60fed75403dbc1420a12e875e04, State: Initialized, Role: FOLLOWER
I20260811 01:57:41.971872 14427 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:41.972163 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:41.972250 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:41.972270 14427 sys_catalog.cc:565] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:41.974205 14445 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:41.974251 14445 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: loading cluster ID for follower catalog manager: success
I20260811 01:57:41.975384 14445 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: acquiring CA information for follower catalog manager: success
I20260811 01:57:41.975950 14445 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:57:42.027184 14201 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:42.027410 14072 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:42.028466 14378 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" instance_seqno: 1786413460496963) as {username='slave'} at 127.0.0.1:50534; Asking this server to re-register.
I20260811 01:57:42.028472 14377 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" instance_seqno: 1786413460464484) as {username='slave'} at 127.0.0.1:50542; Asking this server to re-register.
I20260811 01:57:42.028574 13911 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:42.028906 14201 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.029026 14072 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.029352 14377 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" instance_seqno: 1786413460419076) as {username='slave'} at 127.0.0.1:50552; Asking this server to re-register.
I20260811 01:57:42.029670 14378 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:42.029704 14374 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:42.030073 13911 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.030679 14374 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:42.031653 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.073475609s
I20260811 01:57:42.138413 14324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:42.138545 14324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.138862 14324 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
I20260811 01:57:42.139180 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc" is_pre_election: true
I20260811 01:57:42.139317 13596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 2.
I20260811 01:57:42.139597 13656 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc; no voters:
I20260811 01:57:42.139798 14324 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:57:42.139856 14324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:42.139901 14324 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:42.140766 14324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.141125 14324 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 3 election: Requested vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
I20260811 01:57:42.141397 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc"
I20260811 01:57:42.141494 13596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:42.142329 14392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04" is_pre_election: true
I20260811 01:57:42.142392 14391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04"
I20260811 01:57:42.142459 14392 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 1.
I20260811 01:57:42.142338 13596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 3.
I20260811 01:57:42.142839 13656 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc; no voters: ba2ce60fed75403dbc1420a12e875e04
I20260811 01:57:42.142995 14324 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:57:42.143190 14324 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 LEADER]: Becoming Leader. State: Replica: 6a4c4821171c424fb6bcb1b00ebf137c, State: Running, Role: LEADER
I20260811 01:57:42.143326 14324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.143894 14450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.143997 14450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.144279 14453 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:42.145342 14453 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]
I20260811 01:57:42.146159 14453 tablet_loader.cc:96] loaded metadata for tablet 2310d0060d5e49c1b73c635b530d1d2f (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.146373 14453 tablet_loader.cc:96] loaded metadata for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.146554 14453 tablet_loader.cc:96] loaded metadata for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.146646 14453 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:42.147027 14453 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:42.147069 14453 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:42.147562 14453 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:42.147996 14453 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Loaded TSK: 0
I20260811 01:57:42.148415 14453 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:42.231294 14391 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 1 FOLLOWER]: Advancing to term 3
I20260811 01:57:42.232419 14391 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 3 FOLLOWER]: Refusing update from remote peer 6a4c4821171c424fb6bcb1b00ebf137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20260811 01:57:42.232843 14450 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260811 01:57:42.233820 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.233937 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.234653 14324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.234759 14324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.234977 13596 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Refusing update from remote peer 6a4c4821171c424fb6bcb1b00ebf137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20260811 01:57:42.235189 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.235297 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.235332 14324 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260811 01:57:42.237665 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.237754 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.237910 14325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.237995 14325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.336966 14069 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.337481 14200 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.338442 13910 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.340272 14324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.340370 14324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.342231 13680 catalog_manager.cc:6264] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd] alter complete (version 1)
I20260811 01:57:42.361032 10906 master.cc:562] Master@127.10.166.188:36879 shutting down...
I20260811 01:57:42.363996 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 LEADER]: Raft consensus shutting down.
I20260811 01:57:42.364236 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:42.364372 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: stopping tablet replica
I20260811 01:57:42.377878 10906 master.cc:584] Master@127.10.166.188:36879 shutdown complete.
I20260811 01:57:42.542243 14430 raft_consensus.cc:493] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a4c4821171c424fb6bcb1b00ebf137c)
I20260811 01:57:42.542361 14325 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a4c4821171c424fb6bcb1b00ebf137c)
I20260811 01:57:42.542443 14325 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.542374 14430 raft_consensus.cc:515] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.542809 14325 leader_election.cc:290] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
I20260811 01:57:42.543116 14430 leader_election.cc:290] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
W20260811 01:57:42.545130 14350 leader_election.cc:336] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879): Network error: Client connection negotiation failed: client connection to 127.10.166.188:36879: connect: Connection refused (error 111)
W20260811 01:57:42.545255 13561 leader_election.cc:336] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879): Network error: Client connection negotiation failed: client connection to 127.10.166.188:36879: connect: Connection refused (error 111)
I20260811 01:57:42.547017 14391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9deebe6f8825456ba2e6b67845bb95fc" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04" is_pre_election: true
I20260811 01:57:42.547137 14391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9deebe6f8825456ba2e6b67845bb95fc in term 3.
I20260811 01:57:42.547381 13556 leader_election.cc:304] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9deebe6f8825456ba2e6b67845bb95fc, ba2ce60fed75403dbc1420a12e875e04; no voters: 6a4c4821171c424fb6bcb1b00ebf137c
I20260811 01:57:42.547533 14325 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 01:57:42.547591 14325 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a4c4821171c424fb6bcb1b00ebf137c)
I20260811 01:57:42.547639 14325 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:42.548060 13596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "9deebe6f8825456ba2e6b67845bb95fc" is_pre_election: true
I20260811 01:57:42.548612 14325 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.548802 13596 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2ce60fed75403dbc1420a12e875e04 in current term 4: Already voted for candidate 9deebe6f8825456ba2e6b67845bb95fc in this term.
I20260811 01:57:42.548954 14325 leader_election.cc:290] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 election: Requested vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
I20260811 01:57:42.549172 14391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9deebe6f8825456ba2e6b67845bb95fc" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04"
I20260811 01:57:42.549192 14349 leader_election.cc:304] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2ce60fed75403dbc1420a12e875e04; no voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc
I20260811 01:57:42.549271 14391 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:42.550010 14391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9deebe6f8825456ba2e6b67845bb95fc in term 4.
I20260811 01:57:42.550100 14430 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:57:42.550279 13556 leader_election.cc:304] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9deebe6f8825456ba2e6b67845bb95fc, ba2ce60fed75403dbc1420a12e875e04; no voters:
I20260811 01:57:42.550460 14325 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Leader election won for term 4
I20260811 01:57:42.550536 14325 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 LEADER]: Becoming Leader. State: Replica: 9deebe6f8825456ba2e6b67845bb95fc, State: Running, Role: LEADER
I20260811 01:57:42.550673 14325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
W20260811 01:57:42.550892 13561 leader_election.cc:336] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879): Network error: Client connection negotiation failed: client connection to 127.10.166.188:36879: connect: Connection refused (error 111)
I20260811 01:57:42.551185 14325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9deebe6f8825456ba2e6b67845bb95fc. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.551260 14325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.551503 14486 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:42.552299 14486 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]
I20260811 01:57:42.552860 14486 tablet_loader.cc:96] loaded metadata for tablet 2310d0060d5e49c1b73c635b530d1d2f (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.553038 14486 tablet_loader.cc:96] loaded metadata for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.553176 14486 tablet_loader.cc:96] loaded metadata for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.553252 14486 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:42.553597 14486 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:42.553639 14486 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:42.554101 14486 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:42.554471 14486 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Loaded TSK: 0
I20260811 01:57:42.554646 14486 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:42.581063 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:42.582593 14494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:42.582695 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:42.582649 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:42.582875 14500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:42.583083 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:42.583132 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:42.583153 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413462583152 us; error 0 us; skew 500 ppm
I20260811 01:57:42.584192 10906 webserver.cc:533] Webserver started at http://127.10.166.188:43357/ using document root <none> and password file <none>
I20260811 01:57:42.584357 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:42.584398 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:42.585429 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:42.586190 14507 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:42.586431 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:42.586499 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
uuid: "6a4c4821171c424fb6bcb1b00ebf137c"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:42.586560 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:42.602312 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:42.602597 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:42.606796 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.188:36879
I20260811 01:57:42.607375 14588 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.188:36879 every 8 connection(s)
I20260811 01:57:42.608615 14590 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:57:42.609306 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Bootstrap starting.
I20260811 01:57:42.618790 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:42.619061 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Bootstrap complete.
I20260811 01:57:42.619509 14590 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.619603 14590 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4c4821171c424fb6bcb1b00ebf137c, State: Initialized, Role: FOLLOWER
I20260811 01:57:42.619745 14590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.620033 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.620121 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.620143 14590 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:42.622030 14612 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:42.622074 14612 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: loading cluster ID for follower catalog manager: success
I20260811 01:57:42.623189 14612 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: acquiring CA information for follower catalog manager: success
I20260811 01:57:42.623755 14612 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:57:42.627806 14391 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Refusing update from remote peer 9deebe6f8825456ba2e6b67845bb95fc: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20260811 01:57:42.628160 14460 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260811 01:57:42.628911 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9deebe6f8825456ba2e6b67845bb95fc. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.629012 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.629701 14325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.629797 14325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.630327 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.630419 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.633073 14557 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:42.633903 14557 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Refusing update from remote peer 9deebe6f8825456ba2e6b67845bb95fc: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20260811 01:57:42.634274 14325 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260811 01:57:42.635354 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9deebe6f8825456ba2e6b67845bb95fc. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.635386 14624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "9deebe6f8825456ba2e6b67845bb95fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.635463 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.635489 14624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.649037 14200 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:42.649665 14069 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:42.649729 14541 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" instance_seqno: 1786413460496963) as {username='slave'} at 127.0.0.1:60376; Asking this server to re-register.
I20260811 01:57:42.649837 13910 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:42.650068 14200 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.650374 14541 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" instance_seqno: 1786413460464484) as {username='slave'} at 127.0.0.1:60360; Asking this server to re-register.
I20260811 01:57:42.650437 14539 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" instance_seqno: 1786413460419076) as {username='slave'} at 127.0.0.1:60380; Asking this server to re-register.
I20260811 01:57:42.650758 13910 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.650797 14539 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:42.650758 14069 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.651517 14539 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:42.651597 14541 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:42.652292 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.044852136s
I20260811 01:57:42.652694 13912 heartbeater.cc:499] Master 127.10.166.189:38757 was elected leader, sending a full tablet report...
I20260811 01:57:42.653079 14075 heartbeater.cc:499] Master 127.10.166.189:38757 was elected leader, sending a full tablet report...
I20260811 01:57:42.653712 14202 heartbeater.cc:499] Master 127.10.166.189:38757 was elected leader, sending a full tablet report...
I20260811 01:57:42.656636 10906 master.cc:562] Master@127.10.166.189:38757 shutting down...
I20260811 01:57:42.659409 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 LEADER]: Raft consensus shutting down.
I20260811 01:57:42.659595 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:42.659696 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: stopping tablet replica
I20260811 01:57:42.662653 10906 master.cc:584] Master@127.10.166.189:38757 shutdown complete.
I20260811 01:57:42.830428 14430 raft_consensus.cc:493] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9deebe6f8825456ba2e6b67845bb95fc)
I20260811 01:57:42.830570 14430 raft_consensus.cc:515] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.830945 14430 leader_election.cc:290] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757), 6a4c4821171c424fb6bcb1b00ebf137c (127.10.166.188:36879)
W20260811 01:57:42.832024 14349 leader_election.cc:336] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757): Network error: Client connection negotiation failed: client connection to 127.10.166.189:38757: connect: Connection refused (error 111)
I20260811 01:57:42.834233 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba2ce60fed75403dbc1420a12e875e04" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" is_pre_election: true
I20260811 01:57:42.834589 14350 leader_election.cc:304] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2ce60fed75403dbc1420a12e875e04; no voters: 6a4c4821171c424fb6bcb1b00ebf137c, 9deebe6f8825456ba2e6b67845bb95fc
I20260811 01:57:42.834744 14430 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 01:57:42.835227 14624 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9deebe6f8825456ba2e6b67845bb95fc)
I20260811 01:57:42.835321 14624 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.835772 14624 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
W20260811 01:57:42.836966 14520 leader_election.cc:336] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757): Network error: Client connection negotiation failed: client connection to 127.10.166.189:38757: connect: Connection refused (error 111)
I20260811 01:57:42.838709 14391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04" is_pre_election: true
I20260811 01:57:42.838812 14391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 4.
I20260811 01:57:42.839051 14514 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, ba2ce60fed75403dbc1420a12e875e04; no voters: 9deebe6f8825456ba2e6b67845bb95fc
I20260811 01:57:42.839192 14624 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 01:57:42.839251 14624 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9deebe6f8825456ba2e6b67845bb95fc)
I20260811 01:57:42.839282 14624 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:42.840226 14624 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.840536 14624 leader_election.cc:290] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 election: Requested vote from peers ba2ce60fed75403dbc1420a12e875e04 (127.10.166.190:32923), 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757)
I20260811 01:57:42.840814 14391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "ba2ce60fed75403dbc1420a12e875e04"
I20260811 01:57:42.840925 14391 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 4 FOLLOWER]: Advancing to term 5
W20260811 01:57:42.841426 14520 leader_election.cc:336] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9deebe6f8825456ba2e6b67845bb95fc (127.10.166.189:38757): Network error: Client connection negotiation failed: client connection to 127.10.166.189:38757: connect: Connection refused (error 111)
I20260811 01:57:42.841770 14391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4c4821171c424fb6bcb1b00ebf137c in term 5.
I20260811 01:57:42.842007 14514 leader_election.cc:304] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a4c4821171c424fb6bcb1b00ebf137c, ba2ce60fed75403dbc1420a12e875e04; no voters: 9deebe6f8825456ba2e6b67845bb95fc
I20260811 01:57:42.842163 14624 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 5 FOLLOWER]: Leader election won for term 5
I20260811 01:57:42.842249 14624 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 5 LEADER]: Becoming Leader. State: Replica: 6a4c4821171c424fb6bcb1b00ebf137c, State: Running, Role: LEADER
I20260811 01:57:42.842391 14624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.842897 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.843009 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.843293 14631 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:42.844144 14631 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]
I20260811 01:57:42.844719 14631 tablet_loader.cc:96] loaded metadata for tablet 2310d0060d5e49c1b73c635b530d1d2f (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.844887 14631 tablet_loader.cc:96] loaded metadata for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.845028 14631 tablet_loader.cc:96] loaded metadata for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd])
I20260811 01:57:42.845106 14631 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:42.845461 14631 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:42.845502 14631 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:42.845956 14631 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:42.846321 14631 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c: Loaded TSK: 0
I20260811 01:57:42.846503 14631 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:42.865245 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:42.866660 14632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:42.866699 14633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:42.866775 14635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:42.866827 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:42.867051 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:42.867097 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:42.867112 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413462867112 us; error 0 us; skew 500 ppm
I20260811 01:57:42.868117 10906 webserver.cc:533] Webserver started at http://127.10.166.189:40561/ using document root <none> and password file <none>
I20260811 01:57:42.868275 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:42.868310 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:42.869278 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:42.870090 14643 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:42.870311 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:42.870376 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
uuid: "9deebe6f8825456ba2e6b67845bb95fc"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:42.870429 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:42.879563 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:42.879863 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:42.883610 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.189:38757
I20260811 01:57:42.883671 14731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.189:38757 every 8 connection(s)
I20260811 01:57:42.885035 14732 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:57:42.885717 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Bootstrap starting.
I20260811 01:57:42.895530 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:42.895934 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: Bootstrap complete.
I20260811 01:57:42.896401 14732 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.896498 14732 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9deebe6f8825456ba2e6b67845bb95fc, State: Initialized, Role: FOLLOWER
I20260811 01:57:42.896622 14732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } }
I20260811 01:57:42.896893 14736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.896983 14736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.897046 14732 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:42.899374 14750 catalog_manager.cc:1295] Loaded cluster ID: f4eed7edac184c139e5babf3e002cc40
I20260811 01:57:42.899415 14750 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: loading cluster ID for follower catalog manager: success
I20260811 01:57:42.900549 14750 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: acquiring CA information for follower catalog manager: success
I20260811 01:57:42.901077 14750 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:57:42.956244 13910 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.956431 14687 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:42.957656 14687 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 5 FOLLOWER]: Refusing update from remote peer 6a4c4821171c424fb6bcb1b00ebf137c: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 5 index: 19. (index mismatch)
I20260811 01:57:42.958343 14069 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.958356 14593 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260811 01:57:42.958737 14200 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:42.959530 14736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.959627 14736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.961410 14075 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:42.961705 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.961812 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:42.962083 14674 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" instance_seqno: 1786413460464484) as {username='slave'} at 127.0.0.1:35134; Asking this server to re-register.
I20260811 01:57:42.962487 14075 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.962828 14391 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 5 FOLLOWER]: Refusing update from remote peer 6a4c4821171c424fb6bcb1b00ebf137c: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 5 index: 19. (index mismatch)
I20260811 01:57:42.963253 14674 ts_manager.cc:194] Registered new tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:42.963943 13912 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:42.964187 14736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.964314 14736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.964200 14624 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260811 01:57:42.964556 14674 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" instance_seqno: 1786413460419076) as {username='slave'} at 127.0.0.1:35128; Asking this server to re-register.
I20260811 01:57:42.964661 14202 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:42.964910 13912 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.965430 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c4821171c424fb6bcb1b00ebf137c. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.965521 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.965595 14430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:42.965636 14672 ts_manager.cc:194] Registered new tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:42.965651 14430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:57:42.966005 14674 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" instance_seqno: 1786413460496963) as {username='slave'} at 127.0.0.1:35138; Asking this server to re-register.
I20260811 01:57:42.966317 14202 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:42.966970 14674 ts_manager.cc:194] Registered new tserver with Master: 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:42.968036 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.083957366s
I20260811 01:57:42.968461 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:42.974201 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:42.974386 10906 tablet_replica.cc:333] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: stopping tablet replica
W20260811 01:57:42.974402 14100 consensus_peers.cc:597] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005): Couldn't send request to peer d04f1bc272f941f696a322d68fa2dddf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:42.974516 10906 raft_consensus.cc:2243] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 LEADER]: Raft consensus shutting down.
I20260811 01:57:42.974675 10906 raft_consensus.cc:2272] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:42.975065 10906 tablet_replica.cc:333] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: stopping tablet replica
I20260811 01:57:42.975149 10906 raft_consensus.cc:2243] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:42.975244 10906 raft_consensus.cc:2272] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:42.975585 10906 tablet_replica.cc:333] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: stopping tablet replica
I20260811 01:57:42.975658 10906 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:42.975771 10906 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:42.988477 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:42.990813 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260811 01:57:42.991156 10906 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260811 01:57:42.992404 14762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:42.992488 14761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:42.992498 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:42.992527 14765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:42.992843 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:42.992885 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:42.992899 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413462992899 us; error 0 us; skew 500 ppm
I20260811 01:57:42.993726 10906 webserver.cc:533] Webserver started at http://127.10.166.129:43867/ using document root <none> and password file <none>
I20260811 01:57:42.993854 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:42.993884 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:42.994776 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:42.995623 14773 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:42.995846 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:42.995909 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "d04f1bc272f941f696a322d68fa2dddf"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:42.995985 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.006094 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.006364 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.006603 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:43.007210 14786 ts_tablet_manager.cc:542] Loading tablet metadata (0/3 complete)
I20260811 01:57:43.009562 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260811 01:57:43.009610 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 01:57:43.009650 10906 ts_tablet_manager.cc:600] Registering tablets (0/3 complete)
I20260811 01:57:43.010375 14786 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:43.011176 10906 ts_tablet_manager.cc:616] Registered 3 tablets
I20260811 01:57:43.011219 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 01:57:43.016204 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:42005
I20260811 01:57:43.016253 14885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:42005 every 8 connection(s)
I20260811 01:57:43.018625 14786 tablet.cc:1724] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.018812 14786 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.019093 14786 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=136 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.019577 14786 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Bootstrap complete.
I20260811 01:57:43.020936 14786 ts_tablet_manager.cc:1403] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 01:57:43.021329 14786 raft_consensus.cc:359] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.021431 14786 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.021566 14786 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.021915 14786 ts_tablet_manager.cc:1434] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:43.022078 14786 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:43.026253 14888 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:43.026420 14888 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.026768 14888 heartbeater.cc:507] Master 127.10.166.189:38757 requested a full tablet report, sending...
I20260811 01:57:43.028038 14672 ts_manager.cc:194] Re-registered known tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.028558 14886 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:43.028628 14886 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.028806 14886 heartbeater.cc:507] Master 127.10.166.188:36879 requested a full tablet report, sending...
I20260811 01:57:43.029109 14887 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:43.029232 14786 tablet.cc:1724] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.029300 14887 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.029597 14887 heartbeater.cc:507] Master 127.10.166.190:32923 requested a full tablet report, sending...
I20260811 01:57:43.029619 14541 ts_manager.cc:194] Re-registered known tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.030047 14541 catalog_manager.cc:5811] TS d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005) does not have the latest schema for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]). Expected version 1 got 0
I20260811 01:57:43.030131 14786 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.030380 14786 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=151 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.030669 14786 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap complete.
I20260811 01:57:43.031617 14541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60396
I20260811 01:57:43.032469 14786 ts_tablet_manager.cc:1403] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:57:43.032876 14886 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:43.032989 14786 raft_consensus.cc:359] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.033138 14786 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.033277 14786 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.033457 14378 ts_manager.cc:194] Re-registered known tserver with Master: d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.034132 14786 ts_tablet_manager.cc:1434] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:43.034333 14786 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap starting.
I20260811 01:57:43.040719 14786 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.040836 14786 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.041103 14786 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.041083 14593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2ce60fed75403dbc1420a12e875e04" member_type: VOTER last_known_addr { host: "127.10.166.190" port: 32923 } } peers { permanent_uuid: "9deebe6f8825456ba2e6b67845bb95fc" member_type: VOTER last_known_addr { host: "127.10.166.189" port: 38757 } } peers { permanent_uuid: "6a4c4821171c424fb6bcb1b00ebf137c" member_type: VOTER last_known_addr { host: "127.10.166.188" port: 36879 } } }
I20260811 01:57:43.041239 14593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:43.041420 14786 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Bootstrap complete.
I20260811 01:57:43.042519 14786 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 01:57:43.042850 14786 raft_consensus.cc:359] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.042932 14786 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f1bc272f941f696a322d68fa2dddf, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.043027 14786 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.043242 14786 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:43.099906 14303 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:43.100021 14303 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.100386 14303 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.100706 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:43.100708 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:43.100867 14838 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 3.
I20260811 01:57:43.101140 13941 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde
I20260811 01:57:43.101321 14303 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 01:57:43.101392 14303 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:43.101425 14303 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:43.102257 14303 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.102566 14303 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 4 election: Requested vote from peers 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.102849 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:43.102849 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde"
I20260811 01:57:43.102988 14838 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:43.103935 14838 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in term 4.
I20260811 01:57:43.104185 13941 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: 8f97d94fcabc42b89b54eb018bbf9fde
I20260811 01:57:43.104354 14303 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Leader election won for term 4
I20260811 01:57:43.104519 14303 raft_consensus.cc:697] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 LEADER]: Becoming Leader. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Running, Role: LEADER
I20260811 01:57:43.104650 14303 consensus_queue.cc:237] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.106060 14541 catalog_manager.cc:5719] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a reported cstate change: term changed from 3 to 4, leader changed from d04f1bc272f941f696a322d68fa2dddf (127.10.166.129) to fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130). New cstate: current_term: 4 leader_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:43.171880 14907 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d04f1bc272f941f696a322d68fa2dddf)
I20260811 01:57:43.172011 14907 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.172430 14907 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.172739 14016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:43.172756 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:43.172914 14838 raft_consensus.cc:2393] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f97d94fcabc42b89b54eb018bbf9fde in current term 4: Already voted for candidate fc7d76b1ea514ec18d0cc66531ce6d0a in this term.
I20260811 01:57:43.173144 14100 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde; no voters: d04f1bc272f941f696a322d68fa2dddf, fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:43.173318 14907 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:57:43.174301 14907 raft_consensus.cc:2749] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:57:43.189122 14155 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Refusing update from remote peer fc7d76b1ea514ec18d0cc66531ce6d0a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 01:57:43.189469 14303 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:43.191982 14838 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 4 FOLLOWER]: Refusing update from remote peer fc7d76b1ea514ec18d0cc66531ce6d0a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 01:57:43.192658 14303 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:43.217352 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:43.222077 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:43.222240 10906 tablet_replica.cc:333] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: stopping tablet replica
I20260811 01:57:43.222339 10906 raft_consensus.cc:2243] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.222512 10906 raft_consensus.cc:2272] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.222992 10906 tablet_replica.cc:333] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: stopping tablet replica
I20260811 01:57:43.223073 10906 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.223170 10906 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.223502 10906 tablet_replica.cc:333] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: stopping tablet replica
I20260811 01:57:43.223572 10906 raft_consensus.cc:2243] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.223739 10906 raft_consensus.cc:2272] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.236739 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:43.238873 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260811 01:57:43.239221 10906 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260811 01:57:43.240530 14926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.240540 14930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.240736 14928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.240818 14097 consensus_peers.cc:597] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917): Couldn't send request to peer fc7d76b1ea514ec18d0cc66531ce6d0a. Status: Network error: Client connection negotiation failed: client connection to 127.10.166.130:39917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:43.241019 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:43.241151 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:43.241186 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:43.241207 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413463241207 us; error 0 us; skew 500 ppm
I20260811 01:57:43.242101 10906 webserver.cc:533] Webserver started at http://127.10.166.130:42791/ using document root <none> and password file <none>
I20260811 01:57:43.242252 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:43.242300 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:43.243232 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:43.244227 14937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.244436 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:43.244522 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
format_stamp: "Formatted at 2026-08-11 01:57:40 on dist-test-slave-1l3l"
I20260811 01:57:43.244587 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.251893 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.252156 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.252404 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:43.255244 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260811 01:57:43.255326 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 01:57:43.256160 14949 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:43.256958 10906 ts_tablet_manager.cc:616] Registered 3 tablets
I20260811 01:57:43.257007 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:43.261684 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:39917
I20260811 01:57:43.261801 15046 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:39917 every 8 connection(s)
I20260811 01:57:43.262877 14949 tablet.cc:1724] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.263083 14949 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.263648 14949 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=136 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.264060 14949 tablet_bootstrap.cc:492] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap complete.
I20260811 01:57:43.265617 14949 ts_tablet_manager.cc:1403] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:57:43.266049 14949 raft_consensus.cc:359] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.266156 14949 raft_consensus.cc:740] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.266291 14949 consensus_queue.cc:260] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.266647 14949 ts_tablet_manager.cc:1434] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:43.266811 14949 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:43.267441 15048 heartbeater.cc:344] Connected to a master server at 127.10.166.188:36879
I20260811 01:57:43.267437 15049 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32923
I20260811 01:57:43.267545 15048 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.267562 15049 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.267757 15050 heartbeater.cc:344] Connected to a master server at 127.10.166.189:38757
I20260811 01:57:43.267812 15049 heartbeater.cc:507] Master 127.10.166.190:32923 requested a full tablet report, sending...
I20260811 01:57:43.267843 15050 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.267874 15048 heartbeater.cc:507] Master 127.10.166.188:36879 requested a full tablet report, sending...
I20260811 01:57:43.268003 15050 heartbeater.cc:507] Master 127.10.166.189:38757 requested a full tablet report, sending...
I20260811 01:57:43.268648 14378 ts_manager.cc:194] Re-registered known tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:43.268731 14541 ts_manager.cc:194] Re-registered known tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:43.268812 14672 ts_manager.cc:194] Re-registered known tserver with Master: fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:43.269773 14541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60404
I20260811 01:57:43.270567 15048 heartbeater.cc:499] Master 127.10.166.188:36879 was elected leader, sending a full tablet report...
I20260811 01:57:43.271544 14541 catalog_manager.cc:5811] TS fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917) does not have the latest schema for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]). Expected version 1 got 0
I20260811 01:57:43.274737 14949 tablet.cc:1724] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.274863 14949 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.275146 14949 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=151 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.275425 14949 tablet_bootstrap.cc:492] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap complete.
W20260811 01:57:43.275975 14514 catalog_manager.cc:5129] TS fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917): alter failed for tablet 5b9b8e0a66174c639a28dead93e92e36 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]): Illegal state: Tablet not RUNNING: BOOTSTRAPPING
I20260811 01:57:43.276676 14949 ts_tablet_manager.cc:1403] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:57:43.277033 14949 raft_consensus.cc:359] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.277119 14949 raft_consensus.cc:740] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.277237 14949 consensus_queue.cc:260] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.277493 14949 ts_tablet_manager.cc:1434] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:43.277756 14949 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
I20260811 01:57:43.283948 14949 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.284049 14949 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.284257 14949 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=1 applied=4 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.284495 14949 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap complete.
I20260811 01:57:43.285589 14949 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
E20260811 01:57:43.285974 14949 ts_tablet_manager.cc:1446] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Tablet failed to start: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260811 01:57:43.286032 14949 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.286077 14949 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.286115 14949 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.286181 14949 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.349279 14097 consensus_queue.cc:579] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: The tablet replica hosted on peer fc7d76b1ea514ec18d0cc66531ce6d0a has failed
I20260811 01:57:43.353533 14248 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc7d76b1ea514ec18d0cc66531ce6d0a)
I20260811 01:57:43.353628 14248 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.353996 14248 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:43.354228 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:43.354676 14155 consensus_queue.cc:237] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.355561 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:43.355669 14838 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 01:57:43.355669 15004 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 2.
I20260811 01:57:43.356040 14097 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:43.356055 14248 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:43.356282 14919 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:57:43.356345 14919 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Starting leader election (detected failure of leader fc7d76b1ea514ec18d0cc66531ce6d0a)
I20260811 01:57:43.356396 14919 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:43.357421 14919 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.357561 14907 raft_consensus.cc:2955] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } }
I20260811 01:57:43.357738 14919 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 election: Requested vote from peers d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005), fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917)
I20260811 01:57:43.358085 14837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:43.358157 14838 raft_consensus.cc:2955] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } }
I20260811 01:57:43.359092 14518 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260811 01:57:43.359148 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:43.359238 15004 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:43.360107 15004 raft_consensus.cc:2468] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 3.
I20260811 01:57:43.360438 14097 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters: d04f1bc272f941f696a322d68fa2dddf
I20260811 01:57:43.360839 14541 catalog_manager.cc:5719] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf reported cstate change: config changed from index -1 to 5, VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) evicted. New cstate: current_term: 3 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } }
I20260811 01:57:43.361279 14907 raft_consensus.cc:2804] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:57:43.361361 14907 raft_consensus.cc:697] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Becoming Leader. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:43.361471 14907 consensus_queue.cc:237] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.365507 14155 consensus_queue.cc:237] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } }
I20260811 01:57:43.365684 14990 tablet_service.cc:1558] Processing DeleteTablet for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 with delete_type TABLET_DATA_TOMBSTONED (TS fc7d76b1ea514ec18d0cc66531ce6d0a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60608
I20260811 01:57:43.366134 15072 ts_tablet_manager.cc:1916] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 01:57:43.366637 14838 raft_consensus.cc:1275] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260811 01:57:43.366816 14097 consensus_peers.cc:597] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917): Couldn't send request to peer fc7d76b1ea514ec18d0cc66531ce6d0a. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:43.366909 14539 catalog_manager.cc:5719] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: term changed from 2 to 3, leader changed from fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) to 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131). New cstate: current_term: 3 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:43.367019 14907 consensus_queue.cc:1048] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:57:43.368414 15072 ts_tablet_manager.cc:1929] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260811 01:57:43.368538 15072 log.cc:1199] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/minicluster-data/ts-1-root/wals/ec9a2857d50d48d3bdfcc7c6e0d7fcf0
I20260811 01:57:43.369220 14514 catalog_manager.cc:5050] TS fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917): tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 (table NewTableName [id=3c940178366b45f281a1f6e9f43c2ecd]) successfully deleted
I20260811 01:57:43.369503 14919 raft_consensus.cc:2955] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } } }
I20260811 01:57:43.369936 14838 raft_consensus.cc:2955] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } } }
I20260811 01:57:43.370903 14518 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 with cas_config_opid_index 5: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260811 01:57:43.371824 14539 catalog_manager.cc:5719] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: config changed from index 5 to 6, NON_VOTER fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) added. New cstate: current_term: 3 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:43.392912 14907 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc7d76b1ea514ec18d0cc66531ce6d0a)
I20260811 01:57:43.393002 14907 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.393327 14907 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.393616 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:43.393616 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf" is_pre_election: true
I20260811 01:57:43.393767 14838 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 4.
I20260811 01:57:43.393767 15004 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 4.
I20260811 01:57:43.394044 14097 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:43.394219 14907 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 01:57:43.394276 14907 raft_consensus.cc:493] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Starting leader election (detected failure of leader fc7d76b1ea514ec18d0cc66531ce6d0a)
I20260811 01:57:43.394348 14907 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:43.395049 14907 raft_consensus.cc:515] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.395371 14907 leader_election.cc:290] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 5 election: Requested vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005)
I20260811 01:57:43.395646 14838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d04f1bc272f941f696a322d68fa2dddf"
I20260811 01:57:43.395646 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2310d0060d5e49c1b73c635b530d1d2f" candidate_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a"
I20260811 01:57:43.395798 14838 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:43.395813 15004 raft_consensus.cc:3060] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:57:43.396653 15004 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 5.
I20260811 01:57:43.396653 14838 raft_consensus.cc:2468] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f97d94fcabc42b89b54eb018bbf9fde in term 5.
I20260811 01:57:43.396960 14097 leader_election.cc:304] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a; no voters:
I20260811 01:57:43.397101 14907 raft_consensus.cc:2804] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 5 FOLLOWER]: Leader election won for term 5
I20260811 01:57:43.397195 14907 raft_consensus.cc:697] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 5 LEADER]: Becoming Leader. State: Replica: 8f97d94fcabc42b89b54eb018bbf9fde, State: Running, Role: LEADER
I20260811 01:57:43.397353 14907 consensus_queue.cc:237] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } }
I20260811 01:57:43.398705 14541 catalog_manager.cc:5719] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde reported cstate change: term changed from 4 to 5, leader changed from fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130) to 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131). New cstate: current_term: 5 leader_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:43.414171 14897 raft_consensus.cc:493] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc7d76b1ea514ec18d0cc66531ce6d0a)
I20260811 01:57:43.414256 14897 raft_consensus.cc:515] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } }
I20260811 01:57:43.414657 14897 leader_election.cc:290] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc7d76b1ea514ec18d0cc66531ce6d0a (127.10.166.130:39917), 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:43.417760 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" is_pre_election: true
I20260811 01:57:43.417742 14155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9b8e0a66174c639a28dead93e92e36" candidate_uuid: "d04f1bc272f941f696a322d68fa2dddf" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" is_pre_election: true
I20260811 01:57:43.417909 15004 raft_consensus.cc:2393] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d04f1bc272f941f696a322d68fa2dddf in current term 3: Already voted for candidate 8f97d94fcabc42b89b54eb018bbf9fde in this term.
I20260811 01:57:43.418170 14774 leader_election.cc:304] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d04f1bc272f941f696a322d68fa2dddf; no voters: 8f97d94fcabc42b89b54eb018bbf9fde, fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:43.418354 14897 raft_consensus.cc:3060] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:43.419140 14897 raft_consensus.cc:2749] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/integration-tests/ts_tablet_manager-itest.cc:907: Failure
Failed
Bad status: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260811 01:57:43.463011 10906 tablet_server.cc:179] TabletServer@127.10.166.129:42005 shutting down...
I20260811 01:57:43.466605 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:43.466802 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.466897 10906 raft_consensus.cc:2243] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.467011 10906 raft_consensus.cc:2272] T 2310d0060d5e49c1b73c635b530d1d2f P d04f1bc272f941f696a322d68fa2dddf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.467625 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.467730 10906 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.467837 10906 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.468348 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.468422 10906 raft_consensus.cc:2243] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.468503 10906 raft_consensus.cc:2272] T 5b9b8e0a66174c639a28dead93e92e36 P d04f1bc272f941f696a322d68fa2dddf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.475683 15073 ts_tablet_manager.cc:933] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Initiating tablet copy from peer 8f97d94fcabc42b89b54eb018bbf9fde (127.10.166.131:34463)
I20260811 01:57:43.476236 15073 tablet_copy_client.cc:323] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet copy: Beginning tablet copy session from remote peer at address 127.10.166.131:34463
W20260811 01:57:43.478962 14100 consensus_peers.cc:597] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005): Couldn't send request to peer d04f1bc272f941f696a322d68fa2dddf. Status: Network error: Client connection negotiation failed: client connection to 127.10.166.129:42005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:43.479462 14174 tablet_copy_service.cc:140] P 8f97d94fcabc42b89b54eb018bbf9fde: Received BeginTabletCopySession request for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 from peer fc7d76b1ea514ec18d0cc66531ce6d0a ({username='slave'} at 127.0.0.1:39766)
I20260811 01:57:43.479563 14174 tablet_copy_service.cc:161] P 8f97d94fcabc42b89b54eb018bbf9fde: Beginning new tablet copy session on tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 from peer fc7d76b1ea514ec18d0cc66531ce6d0a at {username='slave'} at 127.0.0.1:39766: session id = fc7d76b1ea514ec18d0cc66531ce6d0a-ec9a2857d50d48d3bdfcc7c6e0d7fcf0
I20260811 01:57:43.480854 14174 tablet_copy_source_session.cc:215] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:57:43.482095 10906 tablet_server.cc:196] TabletServer@127.10.166.129:42005 shutdown complete.
I20260811 01:57:43.482331 15073 ts_tablet_manager.cc:1916] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260811 01:57:43.482824 15004 raft_consensus.cc:1275] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 01:57:43.483179 14907 consensus_queue.cc:1048] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:43.484552 15073 ts_tablet_manager.cc:1929] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 3.4
I20260811 01:57:43.484730 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:43.485172 15004 raft_consensus.cc:1275] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 5 FOLLOWER]: Refusing update from remote peer 8f97d94fcabc42b89b54eb018bbf9fde: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260811 01:57:43.486003 15073 tablet_copy_client.cc:806] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet copy: Starting download of 0 data blocks...
I20260811 01:57:43.486091 14919 consensus_queue.cc:1048] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 01:57:43.486169 15073 tablet_copy_client.cc:670] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet copy: Starting download of 1 WAL segments...
I20260811 01:57:43.489228 15073 tablet_copy_client.cc:538] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:57:43.490540 15073 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap starting.
W20260811 01:57:43.492555 14100 consensus_peers.cc:597] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005): Couldn't send request to peer d04f1bc272f941f696a322d68fa2dddf. Status: Network error: Client connection negotiation failed: client connection to 127.10.166.129:42005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 01:57:43.492632 14100 consensus_peers.cc:597] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde -> Peer d04f1bc272f941f696a322d68fa2dddf (127.10.166.129:42005): Couldn't send request to peer d04f1bc272f941f696a322d68fa2dddf. Status: Network error: Client connection negotiation failed: client connection to 127.10.166.129:42005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:43.492906 10906 tablet_server.cc:179] TabletServer@127.10.166.130:39917 shutting down...
I20260811 01:57:43.497116 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:43.498708 15073 tablet.cc:1724] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 01:57:43.498826 15073 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260811 01:57:43.499487 15073 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:43.499691 15073 tablet_bootstrap.cc:492] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Bootstrap complete.
I20260811 01:57:43.500759 15073 ts_tablet_manager.cc:1403] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.004s
I20260811 01:57:43.501118 15073 raft_consensus.cc:359] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } }
I20260811 01:57:43.501205 15073 raft_consensus.cc:740] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: fc7d76b1ea514ec18d0cc66531ce6d0a, State: Initialized, Role: LEARNER
I20260811 01:57:43.501289 15073 consensus_queue.cc:260] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d04f1bc272f941f696a322d68fa2dddf" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42005 } } peers { permanent_uuid: "8f97d94fcabc42b89b54eb018bbf9fde" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34463 } } peers { permanent_uuid: "fc7d76b1ea514ec18d0cc66531ce6d0a" member_type: NON_VOTER last_known_addr { host: "127.10.166.130" port: 39917 } attrs { promote: true } }
I20260811 01:57:43.501528 15073 ts_tablet_manager.cc:1434] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:43.501967 14174 tablet_copy_service.cc:342] P 8f97d94fcabc42b89b54eb018bbf9fde: Request end of tablet copy session fc7d76b1ea514ec18d0cc66531ce6d0a-ec9a2857d50d48d3bdfcc7c6e0d7fcf0 received from {username='slave'} at 127.0.0.1:39766
I20260811 01:57:43.502102 14174 tablet_copy_service.cc:434] P 8f97d94fcabc42b89b54eb018bbf9fde: ending tablet copy session fc7d76b1ea514ec18d0cc66531ce6d0a-ec9a2857d50d48d3bdfcc7c6e0d7fcf0 on tablet ec9a2857d50d48d3bdfcc7c6e0d7fcf0 with peer fc7d76b1ea514ec18d0cc66531ce6d0a
I20260811 01:57:43.503310 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.503425 10906 raft_consensus.cc:2243] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.503520 10906 raft_consensus.cc:2272] T 2310d0060d5e49c1b73c635b530d1d2f P fc7d76b1ea514ec18d0cc66531ce6d0a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.504038 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.504112 10906 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 LEARNER]: Raft consensus shutting down.
I20260811 01:57:43.504199 10906 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 LEARNER]: Raft consensus is shut down!
I20260811 01:57:43.504689 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.504760 10906 raft_consensus.cc:2243] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.504855 10906 raft_consensus.cc:2272] T 5b9b8e0a66174c639a28dead93e92e36 P fc7d76b1ea514ec18d0cc66531ce6d0a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.517887 10906 tablet_server.cc:196] TabletServer@127.10.166.130:39917 shutdown complete.
I20260811 01:57:43.519882 10906 tablet_server.cc:179] TabletServer@127.10.166.131:0 shutting down...
I20260811 01:57:43.523437 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:43.523595 10906 tablet_replica.cc:333] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde: stopping tablet replica
I20260811 01:57:43.523682 10906 raft_consensus.cc:2243] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.523905 10906 raft_consensus.cc:2272] T ec9a2857d50d48d3bdfcc7c6e0d7fcf0 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.524349 10906 tablet_replica.cc:333] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde: stopping tablet replica
I20260811 01:57:43.524420 10906 raft_consensus.cc:2243] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.524569 10906 raft_consensus.cc:2272] T 5b9b8e0a66174c639a28dead93e92e36 P 8f97d94fcabc42b89b54eb018bbf9fde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.525079 10906 tablet_replica.cc:333] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde: stopping tablet replica
I20260811 01:57:43.525148 10906 raft_consensus.cc:2243] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 5 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.525297 10906 raft_consensus.cc:2272] T 2310d0060d5e49c1b73c635b530d1d2f P 8f97d94fcabc42b89b54eb018bbf9fde [term 5 FOLLOWER]: Raft consensus is shut down!
W20260811 01:57:43.528015 10906 rolling_log.cc:182] Unable to compress old log file: Not found: Unable to open input file to compress: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0/ts_tablet_manager-itest.dist-test-slave-1l3l.slave.diagnostics.20260811-015740.5.10906: No such file or directory (error 2)
I20260811 01:57:43.528218 10906 tablet_server.cc:196] TabletServer@127.10.166.131:0 shutdown complete.
I20260811 01:57:43.529927 10906 master.cc:562] Master@127.10.166.190:32923 shutting down...
I20260811 01:57:43.532503 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.532614 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ba2ce60fed75403dbc1420a12e875e04 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.532675 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.545145 10906 master.cc:584] Master@127.10.166.190:32923 shutdown complete.
I20260811 01:57:43.547039 10906 master.cc:562] Master@127.10.166.189:38757 shutting down...
I20260811 01:57:43.550158 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 5 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:43.550279 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9deebe6f8825456ba2e6b67845bb95fc [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.550356 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.562664 10906 master.cc:584] Master@127.10.166.189:38757 shutdown complete.
I20260811 01:57:43.564679 10906 master.cc:562] Master@127.10.166.188:36879 shutting down...
I20260811 01:57:43.567451 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 5 LEADER]: Raft consensus shutting down.
I20260811 01:57:43.567638 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6a4c4821171c424fb6bcb1b00ebf137c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:43.567693 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:43.580195 10906 master.cc:584] Master@127.10.166.188:36879 shutdown complete.
I20260811 01:57:43.583179 10906 test_util.cc:182] -----------------------------------------------
I20260811 01:57:43.583231 10906 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1786413426438645-10906-0
[ FAILED ] TsTabletManagerITest.TestTableStats (3294 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260811 01:57:43.584645 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:35161
I20260811 01:57:43.584978 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:43.586337 15087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.586421 15084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.586532 15085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:43.586653 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:43.586822 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:43.586858 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:43.586871 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413463586871 us; error 0 us; skew 500 ppm
I20260811 01:57:43.587560 10906 webserver.cc:533] Webserver started at http://127.10.166.190:42719/ using document root <none> and password file <none>
I20260811 01:57:43.587694 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:43.587764 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:43.587836 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:43.588187 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "2871b1960ded4251bac5227184e68190"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.589537 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:43.590355 15097 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.590569 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:43.590634 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "2871b1960ded4251bac5227184e68190"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.590704 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.627699 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.628104 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.631970 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:35161
I20260811 01:57:43.635690 15178 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:35161 every 8 connection(s)
I20260811 01:57:43.636183 15179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:43.637887 15179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: Bootstrap starting.
I20260811 01:57:43.638662 15179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:43.639616 15179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: No bootstrap required, opened a new log
I20260811 01:57:43.640027 15179 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER }
I20260811 01:57:43.640113 15179 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:43.640143 15179 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2871b1960ded4251bac5227184e68190, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.640280 15179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER }
I20260811 01:57:43.640363 15179 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:43.640404 15179 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:43.640451 15179 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:43.641119 15179 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER }
I20260811 01:57:43.641239 15179 leader_election.cc:304] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2871b1960ded4251bac5227184e68190; no voters:
I20260811 01:57:43.641402 15179 leader_election.cc:290] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:43.641506 15189 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:43.641703 15189 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 1 LEADER]: Becoming Leader. State: Replica: 2871b1960ded4251bac5227184e68190, State: Running, Role: LEADER
I20260811 01:57:43.641812 15179 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:43.641846 15189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER }
I20260811 01:57:43.642273 15191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2871b1960ded4251bac5227184e68190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER } }
I20260811 01:57:43.642315 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2871b1960ded4251bac5227184e68190. Latest consensus state: current_term: 1 leader_uuid: "2871b1960ded4251bac5227184e68190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871b1960ded4251bac5227184e68190" member_type: VOTER } }
I20260811 01:57:43.642453 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:43.642709 15207 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:43.642968 15191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:43.643595 15207 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:43.643757 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:43.645380 15207 catalog_manager.cc:1383] Generated new cluster ID: 9fc1395b5fc14253bd90e3b4cdca44a6
I20260811 01:57:43.645438 15207 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:43.660897 15207 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:43.661413 15207 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:43.669718 15207 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: Generated new TSK 0
I20260811 01:57:43.669869 15207 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:43.675920 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:43.677573 15225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.677634 15228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.677634 15226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:43.677735 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:43.677919 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:43.677963 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:43.677977 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413463677977 us; error 0 us; skew 500 ppm
I20260811 01:57:43.678758 10906 webserver.cc:533] Webserver started at http://127.10.166.129:32787/ using document root <none> and password file <none>
I20260811 01:57:43.678905 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:43.678952 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:43.679021 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:43.679354 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "f8b0b874a02a4cb5a0640af088622683"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.680869 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:43.681770 15235 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.681995 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:43.682055 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "f8b0b874a02a4cb5a0640af088622683"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.682121 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.696529 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.696825 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.697069 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:43.697481 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:43.697518 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.697558 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:43.697587 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.701484 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:42975
I20260811 01:57:43.701514 15333 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:42975 every 8 connection(s)
I20260811 01:57:43.702194 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:43.703626 15340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.703778 15344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:43.703913 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:43.703919 15339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:43.704246 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:43.704291 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:43.704314 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413463704310 us; error 0 us; skew 500 ppm
I20260811 01:57:43.705072 10906 webserver.cc:533] Webserver started at http://127.10.166.130:44703/ using document root <none> and password file <none>
I20260811 01:57:43.705211 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:43.705267 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:43.705335 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:43.705729 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "7621a8cf990c4d84928fb453ac509926"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.706688 15335 heartbeater.cc:344] Connected to a master server at 127.10.166.190:35161
I20260811 01:57:43.706784 15335 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.706991 15335 heartbeater.cc:507] Master 127.10.166.190:35161 requested a full tablet report, sending...
I20260811 01:57:43.707425 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:43.707597 15123 ts_manager.cc:194] Registered new tserver with Master: f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975)
I20260811 01:57:43.708376 15352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.708415 15123 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49478
I20260811 01:57:43.708629 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:43.708701 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "7621a8cf990c4d84928fb453ac509926"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.708770 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.721211 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.721472 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.721701 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:43.722077 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:43.722112 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.722144 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:43.722172 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.725960 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:39393
I20260811 01:57:43.726018 15457 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:39393 every 8 connection(s)
I20260811 01:57:43.726662 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:43.728120 15465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.728204 15466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:43.728237 15469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:43.728621 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:43.728797 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:43.728845 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:43.728873 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413463728873 us; error 0 us; skew 500 ppm
I20260811 01:57:43.729664 10906 webserver.cc:533] Webserver started at http://127.10.166.131:37849/ using document root <none> and password file <none>
I20260811 01:57:43.729806 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:43.729851 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:43.729900 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:43.730278 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root/instance:
uuid: "8ad4437e4ff3420b88d453c566f8b134"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.730597 15458 heartbeater.cc:344] Connected to a master server at 127.10.166.190:35161
I20260811 01:57:43.730681 15458 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.730856 15458 heartbeater.cc:507] Master 127.10.166.190:35161 requested a full tablet report, sending...
I20260811 01:57:43.731432 15123 ts_manager.cc:194] Registered new tserver with Master: 7621a8cf990c4d84928fb453ac509926 (127.10.166.130:39393)
I20260811 01:57:43.731940 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:43.732048 15123 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49484
I20260811 01:57:43.732903 15478 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.733104 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:43.733163 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root
uuid: "8ad4437e4ff3420b88d453c566f8b134"
format_stamp: "Formatted at 2026-08-11 01:57:43 on dist-test-slave-1l3l"
I20260811 01:57:43.733222 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:43.748679 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:43.748982 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:43.749220 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:43.749632 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:43.749670 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.749711 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:43.749739 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:43.754052 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.131:46477
I20260811 01:57:43.754102 15578 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.131:46477 every 8 connection(s)
I20260811 01:57:43.757869 15579 heartbeater.cc:344] Connected to a master server at 127.10.166.190:35161
I20260811 01:57:43.757961 15579 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:43.758117 15579 heartbeater.cc:507] Master 127.10.166.190:35161 requested a full tablet report, sending...
I20260811 01:57:43.758644 15123 ts_manager.cc:194] Registered new tserver with Master: 8ad4437e4ff3420b88d453c566f8b134 (127.10.166.131:46477)
I20260811 01:57:43.758931 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004494206s
I20260811 01:57:43.759121 15123 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49496
I20260811 01:57:43.768688 15123 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260811 01:57:43.769074 15123 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:57:43.777354 15400 tablet_service.cc:1511] Processing CreateTablet for tablet 7da1a0430f2541019f0858836b529d70 (DEFAULT_TABLE table=test-table [id=f04c0f12e3f14911a7347867de65f3fb]), partition=
I20260811 01:57:43.777598 15400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da1a0430f2541019f0858836b529d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:43.778769 15277 tablet_service.cc:1511] Processing CreateTablet for tablet 7da1a0430f2541019f0858836b529d70 (DEFAULT_TABLE table=test-table [id=f04c0f12e3f14911a7347867de65f3fb]), partition=
I20260811 01:57:43.779011 15277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da1a0430f2541019f0858836b529d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:43.779104 15523 tablet_service.cc:1511] Processing CreateTablet for tablet 7da1a0430f2541019f0858836b529d70 (DEFAULT_TABLE table=test-table [id=f04c0f12e3f14911a7347867de65f3fb]), partition=
I20260811 01:57:43.779320 15523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da1a0430f2541019f0858836b529d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:43.781038 15604 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Bootstrap starting.
I20260811 01:57:43.781344 15607 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Bootstrap starting.
I20260811 01:57:43.782024 15604 tablet_bootstrap.cc:654] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:43.782261 15607 tablet_bootstrap.cc:654] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:43.782893 15603 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Bootstrap starting.
I20260811 01:57:43.783274 15604 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: No bootstrap required, opened a new log
I20260811 01:57:43.783358 15604 ts_tablet_manager.cc:1403] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:43.783767 15607 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: No bootstrap required, opened a new log
I20260811 01:57:43.783860 15603 tablet_bootstrap.cc:654] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:43.783871 15607 ts_tablet_manager.cc:1403] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:57:43.783807 15604 raft_consensus.cc:359] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.783986 15604 raft_consensus.cc:385] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:43.784034 15604 raft_consensus.cc:740] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b0b874a02a4cb5a0640af088622683, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.784168 15604 consensus_queue.cc:260] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.784312 15607 raft_consensus.cc:359] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.784418 15607 raft_consensus.cc:385] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:43.784458 15607 raft_consensus.cc:740] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad4437e4ff3420b88d453c566f8b134, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.784528 15604 ts_tablet_manager.cc:1434] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 01:57:43.784581 15604 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260811 01:57:43.784726 15335 heartbeater.cc:499] Master 127.10.166.190:35161 was elected leader, sending a full tablet report...
I20260811 01:57:43.784960 15603 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: No bootstrap required, opened a new log
I20260811 01:57:43.785038 15603 ts_tablet_manager.cc:1403] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:43.784608 15607 consensus_queue.cc:260] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.785362 15607 ts_tablet_manager.cc:1434] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 01:57:43.785398 15607 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260811 01:57:43.785449 15603 raft_consensus.cc:359] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.785543 15603 raft_consensus.cc:385] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:43.785573 15603 raft_consensus.cc:740] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7621a8cf990c4d84928fb453ac509926, State: Initialized, Role: FOLLOWER
I20260811 01:57:43.785682 15603 consensus_queue.cc:260] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:43.786021 15603 ts_tablet_manager.cc:1434] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:43.786036 15458 heartbeater.cc:499] Master 127.10.166.190:35161 was elected leader, sending a full tablet report...
W20260811 01:57:43.786077 15603 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260811 01:57:43.786274 15579 heartbeater.cc:499] Master 127.10.166.190:35161 was elected leader, sending a full tablet report...
I20260811 01:57:44.011116 15612 raft_consensus.cc:493] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:44.011253 15612 raft_consensus.cc:515] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:44.011749 15612 leader_election.cc:290] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7621a8cf990c4d84928fb453ac509926 (127.10.166.130:39393), f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975)
I20260811 01:57:44.015311 15413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da1a0430f2541019f0858836b529d70" candidate_uuid: "8ad4437e4ff3420b88d453c566f8b134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7621a8cf990c4d84928fb453ac509926" is_pre_election: true
I20260811 01:57:44.015470 15413 raft_consensus.cc:2468] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ad4437e4ff3420b88d453c566f8b134 in term 0.
I20260811 01:57:44.015574 15292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da1a0430f2541019f0858836b529d70" candidate_uuid: "8ad4437e4ff3420b88d453c566f8b134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b0b874a02a4cb5a0640af088622683" is_pre_election: true
I20260811 01:57:44.015684 15292 raft_consensus.cc:2468] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ad4437e4ff3420b88d453c566f8b134 in term 0.
I20260811 01:57:44.015825 15482 leader_election.cc:304] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7621a8cf990c4d84928fb453ac509926, 8ad4437e4ff3420b88d453c566f8b134; no voters:
I20260811 01:57:44.016010 15612 raft_consensus.cc:2804] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:44.016074 15612 raft_consensus.cc:493] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:44.016124 15612 raft_consensus.cc:3060] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.016984 15612 raft_consensus.cc:515] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:44.017299 15612 leader_election.cc:290] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [CANDIDATE]: Term 1 election: Requested vote from peers 7621a8cf990c4d84928fb453ac509926 (127.10.166.130:39393), f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975)
I20260811 01:57:44.017617 15413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da1a0430f2541019f0858836b529d70" candidate_uuid: "8ad4437e4ff3420b88d453c566f8b134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7621a8cf990c4d84928fb453ac509926"
I20260811 01:57:44.017635 15292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7da1a0430f2541019f0858836b529d70" candidate_uuid: "8ad4437e4ff3420b88d453c566f8b134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b0b874a02a4cb5a0640af088622683"
I20260811 01:57:44.017717 15413 raft_consensus.cc:3060] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.017724 15292 raft_consensus.cc:3060] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.018567 15413 raft_consensus.cc:2468] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ad4437e4ff3420b88d453c566f8b134 in term 1.
I20260811 01:57:44.018617 15292 raft_consensus.cc:2468] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ad4437e4ff3420b88d453c566f8b134 in term 1.
I20260811 01:57:44.018826 15482 leader_election.cc:304] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ad4437e4ff3420b88d453c566f8b134, f8b0b874a02a4cb5a0640af088622683; no voters:
I20260811 01:57:44.018977 15612 raft_consensus.cc:2804] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.019188 15612 raft_consensus.cc:697] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 LEADER]: Becoming Leader. State: Replica: 8ad4437e4ff3420b88d453c566f8b134, State: Running, Role: LEADER
I20260811 01:57:44.019340 15612 consensus_queue.cc:237] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:44.020848 15122 catalog_manager.cc:5719] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ad4437e4ff3420b88d453c566f8b134 (127.10.166.131). New cstate: current_term: 1 leader_uuid: "8ad4437e4ff3420b88d453c566f8b134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:44.072849 15277 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:41466
I20260811 01:57:44.073221 15620 tablet_replica.cc:333] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: stopping tablet replica
I20260811 01:57:44.073347 15620 raft_consensus.cc:2243] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:44.073468 15620 raft_consensus.cc:2272] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.073945 15620 ts_tablet_manager.cc:1916] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 01:57:44.075699 15620 ts_tablet_manager.cc:1929] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260811 01:57:44.075812 15620 log.cc:1199] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root/wals/7da1a0430f2541019f0858836b529d70
I20260811 01:57:44.076951 15621 ts_tablet_manager.cc:933] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Initiating tablet copy from peer 8ad4437e4ff3420b88d453c566f8b134 (127.10.166.131:46477)
I20260811 01:57:44.077253 15621 tablet_copy_client.cc:323] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet copy: Beginning tablet copy session from remote peer at address 127.10.166.131:46477
I20260811 01:57:44.080135 15558 tablet_copy_service.cc:140] P 8ad4437e4ff3420b88d453c566f8b134: Received BeginTabletCopySession request for tablet 7da1a0430f2541019f0858836b529d70 from peer f8b0b874a02a4cb5a0640af088622683 ({username='slave'} at 127.0.0.1:39980)
I20260811 01:57:44.080216 15558 tablet_copy_service.cc:161] P 8ad4437e4ff3420b88d453c566f8b134: Beginning new tablet copy session on tablet 7da1a0430f2541019f0858836b529d70 from peer f8b0b874a02a4cb5a0640af088622683 at {username='slave'} at 127.0.0.1:39980: session id = f8b0b874a02a4cb5a0640af088622683-7da1a0430f2541019f0858836b529d70
I20260811 01:57:44.080869 15558 tablet_copy_source_session.cc:215] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:57:44.081442 15621 ts_tablet_manager.cc:1916] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260811 01:57:44.082906 15621 ts_tablet_manager.cc:1929] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260811 01:57:44.083068 15621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7da1a0430f2541019f0858836b529d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.083990 15621 tablet_copy_client.cc:806] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet copy: Starting download of 0 data blocks...
I20260811 01:57:44.084131 15621 tablet_copy_client.cc:670] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet copy: Starting download of 1 WAL segments...
I20260811 01:57:44.084635 15122 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49504:
table { table_id: "f04c0f12e3f14911a7347867de65f3fb" table_name: "test-table" }
I20260811 01:57:44.084808 15122 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49504:
table { table_id: "f04c0f12e3f14911a7347867de65f3fb" table_name: "test-table" }
I20260811 01:57:44.085148 15621 tablet_copy_client.cc:538] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:57:44.086086 15621 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Bootstrap starting.
I20260811 01:57:44.087126 15122 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: Sending DeleteTablet for 3 replicas of tablet 7da1a0430f2541019f0858836b529d70
I20260811 01:57:44.087764 15523 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:57:44 UTC) from {username='slave'} at 127.0.0.1:39970
I20260811 01:57:44.087764 15400 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:57:44 UTC) from {username='slave'} at 127.0.0.1:39750
I20260811 01:57:44.088047 15277 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:57:44 UTC) from {username='slave'} at 127.0.0.1:41454
I20260811 01:57:44.088097 15626 tablet_replica.cc:333] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: stopping tablet replica
I20260811 01:57:44.088188 15626 raft_consensus.cc:2243] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.088325 15629 tablet_replica.cc:333] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: stopping tablet replica
I20260811 01:57:44.088373 15626 pending_rounds.cc:70] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Trying to abort 1 pending ops.
I20260811 01:57:44.088455 15626 pending_rounds.cc:77] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317149548624629760 op_type: NO_OP noop_request { }
W20260811 01:57:44.088510 15102 catalog_manager.cc:5030] TS f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975): delete failed for tablet 7da1a0430f2541019f0858836b529d70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7da1a0430f2541019f0858836b529d70 already in progress: copying tablet
I20260811 01:57:44.088487 15629 raft_consensus.cc:2243] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:44.088559 15626 raft_consensus.cc:2889] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 01:57:44.088634 15626 raft_consensus.cc:2272] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.088660 15629 raft_consensus.cc:2272] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.089098 15629 ts_tablet_manager.cc:1916] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:57:44.089124 15626 ts_tablet_manager.cc:1916] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:57:44.089162 15621 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 01:57:44.089414 15621 tablet_bootstrap.cc:492] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Bootstrap complete.
I20260811 01:57:44.089564 15621 ts_tablet_manager.cc:1403] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 01:57:44.089964 15621 raft_consensus.cc:359] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:44.090081 15621 raft_consensus.cc:740] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b0b874a02a4cb5a0640af088622683, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.090219 15621 consensus_queue.cc:260] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7621a8cf990c4d84928fb453ac509926" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 39393 } } peers { permanent_uuid: "8ad4437e4ff3420b88d453c566f8b134" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 46477 } } peers { permanent_uuid: "f8b0b874a02a4cb5a0640af088622683" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 42975 } }
I20260811 01:57:44.090495 15621 ts_tablet_manager.cc:1434] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260811 01:57:44.090566 15621 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20260811 01:57:44.090999 15626 ts_tablet_manager.cc:1929] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:57:44.091074 15626 log.cc:1199] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-2-root/wals/7da1a0430f2541019f0858836b529d70
I20260811 01:57:44.091289 15626 ts_tablet_manager.cc:1950] T 7da1a0430f2541019f0858836b529d70 P 8ad4437e4ff3420b88d453c566f8b134: Deleting consensus metadata
I20260811 01:57:44.091799 15102 catalog_manager.cc:5050] TS 8ad4437e4ff3420b88d453c566f8b134 (127.10.166.131:46477): tablet 7da1a0430f2541019f0858836b529d70 (table test-table [id=f04c0f12e3f14911a7347867de65f3fb]) successfully deleted
I20260811 01:57:44.091892 15277 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:57:44 UTC) from {username='slave'} at 127.0.0.1:41454
I20260811 01:57:44.092059 15629 ts_tablet_manager.cc:1929] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 01:57:44.092115 15629 log.cc:1199] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-1-root/wals/7da1a0430f2541019f0858836b529d70
W20260811 01:57:44.092262 15102 catalog_manager.cc:5030] TS f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975): delete failed for tablet 7da1a0430f2541019f0858836b529d70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7da1a0430f2541019f0858836b529d70 already in progress: copying tablet
I20260811 01:57:44.092321 15629 ts_tablet_manager.cc:1950] T 7da1a0430f2541019f0858836b529d70 P 7621a8cf990c4d84928fb453ac509926: Deleting consensus metadata
I20260811 01:57:44.092872 15102 catalog_manager.cc:5050] TS 7621a8cf990c4d84928fb453ac509926 (127.10.166.130:39393): tablet 7da1a0430f2541019f0858836b529d70 (table test-table [id=f04c0f12e3f14911a7347867de65f3fb]) successfully deleted
I20260811 01:57:44.141281 15558 tablet_copy_service.cc:342] P 8ad4437e4ff3420b88d453c566f8b134: Request end of tablet copy session f8b0b874a02a4cb5a0640af088622683-7da1a0430f2541019f0858836b529d70 received from {username='slave'} at 127.0.0.1:39980
I20260811 01:57:44.141383 15558 tablet_copy_service.cc:434] P 8ad4437e4ff3420b88d453c566f8b134: ending tablet copy session f8b0b874a02a4cb5a0640af088622683-7da1a0430f2541019f0858836b529d70 on tablet 7da1a0430f2541019f0858836b529d70 with peer f8b0b874a02a4cb5a0640af088622683
I20260811 01:57:44.142277 15277 tablet_service.cc:1558] Processing DeleteTablet for tablet 7da1a0430f2541019f0858836b529d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:57:44 UTC) from {username='slave'} at 127.0.0.1:41454
I20260811 01:57:44.142493 15620 tablet_replica.cc:333] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: stopping tablet replica
I20260811 01:57:44.142603 15620 raft_consensus.cc:2243] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:44.142697 15620 pending_rounds.cc:70] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Trying to abort 1 pending ops.
I20260811 01:57:44.142727 15620 pending_rounds.cc:77] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317149548624629760 op_type: NO_OP noop_request { }
I20260811 01:57:44.142797 15620 raft_consensus.cc:2889] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 01:57:44.142851 15620 raft_consensus.cc:2272] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.143349 15620 ts_tablet_manager.cc:1916] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:57:44.145064 15620 ts_tablet_manager.cc:1929] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:57:44.145126 15620 log.cc:1199] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1786413426438645-10906-0/minicluster-data/ts-0-root/wals/7da1a0430f2541019f0858836b529d70
I20260811 01:57:44.145337 15620 ts_tablet_manager.cc:1950] T 7da1a0430f2541019f0858836b529d70 P f8b0b874a02a4cb5a0640af088622683: Deleting consensus metadata
I20260811 01:57:44.145913 15102 catalog_manager.cc:5050] TS f8b0b874a02a4cb5a0640af088622683 (127.10.166.129:42975): tablet 7da1a0430f2541019f0858836b529d70 (table test-table [id=f04c0f12e3f14911a7347867de65f3fb]) successfully deleted
I20260811 01:57:44.152446 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:44.155547 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.167672 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:44.169572 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:44.172847 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.184764 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:44.186373 10906 tablet_server.cc:179] TabletServer@127.10.166.131:0 shutting down...
I20260811 01:57:44.189766 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.201781 10906 tablet_server.cc:196] TabletServer@127.10.166.131:0 shutdown complete.
I20260811 01:57:44.203306 10906 master.cc:562] Master@127.10.166.190:35161 shutting down...
I20260811 01:57:44.206084 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.206225 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.206283 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2871b1960ded4251bac5227184e68190: stopping tablet replica
I20260811 01:57:44.208585 10906 master.cc:584] Master@127.10.166.190:35161 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (629 ms)
[----------] 4 tests from TsTabletManagerITest (6561 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260811 01:57:44.213958 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:35863
I20260811 01:57:44.214270 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.215672 15643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.215792 15641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.215806 15646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.215864 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:44.216148 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.216189 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.216207 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464216207 us; error 0 us; skew 500 ppm
I20260811 01:57:44.216908 10906 webserver.cc:533] Webserver started at http://127.10.166.190:36159/ using document root <none> and password file <none>
I20260811 01:57:44.217064 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.217109 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.217177 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.217516 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "24a502dcb3034fc6b72916c97b40e9a5"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.218858 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.219652 15658 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.219882 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.219954 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "24a502dcb3034fc6b72916c97b40e9a5"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.220022 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.233254 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.233529 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.237200 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:35863
I20260811 01:57:44.239527 15731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:35863 every 8 connection(s)
I20260811 01:57:44.239966 15733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.241634 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5: Bootstrap starting.
I20260811 01:57:44.242406 15733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.243321 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5: No bootstrap required, opened a new log
I20260811 01:57:44.243700 15733 raft_consensus.cc:359] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER }
I20260811 01:57:44.243819 15733 raft_consensus.cc:385] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.243852 15733 raft_consensus.cc:740] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a502dcb3034fc6b72916c97b40e9a5, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.243993 15733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER }
I20260811 01:57:44.244076 15733 raft_consensus.cc:399] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.244112 15733 raft_consensus.cc:493] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.244159 15733 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.244823 15733 raft_consensus.cc:515] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER }
I20260811 01:57:44.244947 15733 leader_election.cc:304] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a502dcb3034fc6b72916c97b40e9a5; no voters:
I20260811 01:57:44.245126 15733 leader_election.cc:290] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.245230 15738 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.245440 15738 raft_consensus.cc:697] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 1 LEADER]: Becoming Leader. State: Replica: 24a502dcb3034fc6b72916c97b40e9a5, State: Running, Role: LEADER
I20260811 01:57:44.245546 15733 sys_catalog.cc:565] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:44.245570 15738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER }
I20260811 01:57:44.245994 15742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a502dcb3034fc6b72916c97b40e9a5. Latest consensus state: current_term: 1 leader_uuid: "24a502dcb3034fc6b72916c97b40e9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER } }
I20260811 01:57:44.245971 15739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a502dcb3034fc6b72916c97b40e9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a502dcb3034fc6b72916c97b40e9a5" member_type: VOTER } }
I20260811 01:57:44.246084 15742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.246094 15739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.246773 15747 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:44.247421 15747 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:44.247597 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:44.249163 15747 catalog_manager.cc:1383] Generated new cluster ID: 4e676ba5f04e4ffe8625db606fd63336
I20260811 01:57:44.249220 15747 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:44.257208 15747 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:44.257699 15747 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:44.262872 15747 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5: Generated new TSK 0
I20260811 01:57:44.263022 15747 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:44.279906 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.281761 15768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.281893 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.281821 15770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.281955 15773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.282193 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.282238 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.282258 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464282258 us; error 0 us; skew 500 ppm
I20260811 01:57:44.283062 10906 webserver.cc:533] Webserver started at http://127.10.166.129:35495/ using document root <none> and password file <none>
I20260811 01:57:44.283231 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.283286 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.283366 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.283829 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "5e5035fa1a6540f88334483335d3505e"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.285281 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:44.286335 15781 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.286551 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:44.286629 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "5e5035fa1a6540f88334483335d3505e"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.286710 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.293174 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.293524 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.294338 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:44.294380 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.294423 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:44.294478 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.299748 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:41593
I20260811 01:57:44.299793 15896 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:41593 every 8 connection(s)
I20260811 01:57:44.300832 15898 heartbeater.cc:344] Connected to a master server at 127.10.166.190:35863
I20260811 01:57:44.300930 15898 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.301158 15898 heartbeater.cc:507] Master 127.10.166.190:35863 requested a full tablet report, sending...
I20260811 01:57:44.301766 15684 ts_manager.cc:194] Registered new tserver with Master: 5e5035fa1a6540f88334483335d3505e (127.10.166.129:41593)
I20260811 01:57:44.302526 10906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002254028s
I20260811 01:57:44.303027 15684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50034
I20260811 01:57:44.312573 15835 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 01:57:44.313061 15835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.315578 15912 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Bootstrap starting.
I20260811 01:57:44.316512 15912 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.317411 15912 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: No bootstrap required, opened a new log
I20260811 01:57:44.317488 15912 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:44.317803 15912 raft_consensus.cc:359] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.317883 15912 raft_consensus.cc:385] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.317909 15912 raft_consensus.cc:740] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5035fa1a6540f88334483335d3505e, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.318025 15912 consensus_queue.cc:260] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.318080 15912 raft_consensus.cc:399] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.318107 15912 raft_consensus.cc:493] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.318140 15912 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.318809 15912 raft_consensus.cc:515] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.318929 15912 leader_election.cc:304] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e5035fa1a6540f88334483335d3505e; no voters:
I20260811 01:57:44.319100 15912 leader_election.cc:290] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.319211 15915 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.319406 15898 heartbeater.cc:499] Master 127.10.166.190:35863 was elected leader, sending a full tablet report...
I20260811 01:57:44.319427 15915 raft_consensus.cc:697] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 LEADER]: Becoming Leader. State: Replica: 5e5035fa1a6540f88334483335d3505e, State: Running, Role: LEADER
I20260811 01:57:44.319388 15912 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:44.319641 15915 consensus_queue.cc:237] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.320086 15916 tablet_replica.cc:442] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e5035fa1a6540f88334483335d3505e" committed_config { opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } } }
I20260811 01:57:44.320112 15917 tablet_replica.cc:442] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: TxnStatusTablet state changed. Reason: New leader 5e5035fa1a6540f88334483335d3505e. Latest consensus state: current_term: 1 leader_uuid: "5e5035fa1a6540f88334483335d3505e" committed_config { opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } } }
I20260811 01:57:44.320245 15916 tablet_replica.cc:445] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.320254 15917 tablet_replica.cc:445] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: This TxnStatusTablet replica's current role is: LEADER
W20260811 01:57:44.320789 15684 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.320863 15919 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:57:44.321018 15919 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260811 01:57:44.321919 15684 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.339159 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:44.343308 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.343549 10906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: stopping tablet replica
I20260811 01:57:44.343659 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.343796 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.357152 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:44.360072 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.361781 15926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.361891 15928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.362025 15931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.362147 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:44.362298 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.362341 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.362370 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464362370 us; error 0 us; skew 500 ppm
I20260811 01:57:44.363336 10906 webserver.cc:533] Webserver started at http://127.10.166.129:35495/ using document root <none> and password file <none>
I20260811 01:57:44.363513 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.363557 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:44.364809 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.365793 15941 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.366077 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.366151 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "5e5035fa1a6540f88334483335d3505e"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.366231 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.385103 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.385563 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.386752 15956 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 01:57:44.388199 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 01:57:44.388265 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 01:57:44.388324 10906 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260811 01:57:44.388998 10906 ts_tablet_manager.cc:616] Registered 1 tablets
I20260811 01:57:44.389045 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.389290 15956 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Bootstrap starting.
I20260811 01:57:44.395197 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:41593
I20260811 01:57:44.395349 15956 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:44.395833 15956 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Bootstrap complete.
I20260811 01:57:44.396029 16057 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:41593 every 8 connection(s)
I20260811 01:57:44.396163 15956 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260811 01:57:44.396660 15956 raft_consensus.cc:359] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.396845 15956 raft_consensus.cc:740] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5035fa1a6540f88334483335d3505e, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.396865 16058 heartbeater.cc:344] Connected to a master server at 127.10.166.190:35863
I20260811 01:57:44.397090 15956 consensus_queue.cc:260] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.397455 16058 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.397462 15956 raft_consensus.cc:399] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.397567 15956 raft_consensus.cc:493] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.397655 15956 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:44.397819 16058 heartbeater.cc:507] Master 127.10.166.190:35863 requested a full tablet report, sending...
I20260811 01:57:44.398713 15956 raft_consensus.cc:515] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.398876 15956 leader_election.cc:304] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e5035fa1a6540f88334483335d3505e; no voters:
I20260811 01:57:44.399055 15956 leader_election.cc:290] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:57:44.399338 15956 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 01:57:44.399832 16062 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:44.400017 16062 raft_consensus.cc:697] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 2 LEADER]: Becoming Leader. State: Replica: 5e5035fa1a6540f88334483335d3505e, State: Running, Role: LEADER
I20260811 01:57:44.400159 16062 consensus_queue.cc:237] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } }
I20260811 01:57:44.400549 15684 ts_manager.cc:194] Re-registered known tserver with Master: 5e5035fa1a6540f88334483335d3505e (127.10.166.129:41593)
I20260811 01:57:44.400578 16062 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader 5e5035fa1a6540f88334483335d3505e. Latest consensus state: current_term: 2 leader_uuid: "5e5035fa1a6540f88334483335d3505e" committed_config { opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } } }
I20260811 01:57:44.400651 16062 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
W20260811 01:57:44.400704 15684 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.401245 15684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50048
I20260811 01:57:44.400559 16063 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5e5035fa1a6540f88334483335d3505e" committed_config { opid_index: -1 peers { permanent_uuid: "5e5035fa1a6540f88334483335d3505e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 41593 } } }
I20260811 01:57:44.401562 16063 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.402067 16058 heartbeater.cc:499] Master 127.10.166.190:35863 was elected leader, sending a full tablet report...
W20260811 01:57:44.402781 15684 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.403825 16069 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:57:44.403918 16069 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260811 01:57:44.412242 10906 tablet_server.cc:179] TabletServer@127.10.166.129:41593 shutting down...
I20260811 01:57:44.415885 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.416108 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:44.416227 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 2 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.416343 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 5e5035fa1a6540f88334483335d3505e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.429329 10906 tablet_server.cc:196] TabletServer@127.10.166.129:41593 shutdown complete.
I20260811 01:57:44.432142 10906 master.cc:562] Master@127.10.166.190:35863 shutting down...
I20260811 01:57:44.436302 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.436482 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.436591 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 24a502dcb3034fc6b72916c97b40e9a5: stopping tablet replica
I20260811 01:57:44.439128 10906 master.cc:584] Master@127.10.166.190:35863 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (230 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260811 01:57:44.444461 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:43047
I20260811 01:57:44.444814 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.446300 16080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.446362 16084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.446589 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.447805 16082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.448032 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.448076 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.448091 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464448090 us; error 0 us; skew 500 ppm
I20260811 01:57:44.448875 10906 webserver.cc:533] Webserver started at http://127.10.166.190:39869/ using document root <none> and password file <none>
I20260811 01:57:44.449035 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.449082 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.449182 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.449764 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "a6b163d5d844464b8584a2ccd909937f"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.451366 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.452603 16089 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.453117 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:44.453194 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "a6b163d5d844464b8584a2ccd909937f"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.453267 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.470014 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.470399 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.475126 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:43047
I20260811 01:57:44.475680 16175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:43047 every 8 connection(s)
I20260811 01:57:44.476256 16177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.478691 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f: Bootstrap starting.
I20260811 01:57:44.479779 16177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.480942 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f: No bootstrap required, opened a new log
I20260811 01:57:44.481385 16177 raft_consensus.cc:359] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER }
I20260811 01:57:44.481485 16177 raft_consensus.cc:385] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.481513 16177 raft_consensus.cc:740] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b163d5d844464b8584a2ccd909937f, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.481631 16177 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER }
I20260811 01:57:44.481699 16177 raft_consensus.cc:399] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.481732 16177 raft_consensus.cc:493] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.481774 16177 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.482566 16177 raft_consensus.cc:515] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER }
I20260811 01:57:44.482700 16177 leader_election.cc:304] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6b163d5d844464b8584a2ccd909937f; no voters:
I20260811 01:57:44.482872 16177 leader_election.cc:290] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.482970 16181 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.483206 16181 raft_consensus.cc:697] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 1 LEADER]: Becoming Leader. State: Replica: a6b163d5d844464b8584a2ccd909937f, State: Running, Role: LEADER
I20260811 01:57:44.483265 16177 sys_catalog.cc:565] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:44.483353 16181 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER }
I20260811 01:57:44.483786 16182 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6b163d5d844464b8584a2ccd909937f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER } }
I20260811 01:57:44.483881 16182 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.483803 16190 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6b163d5d844464b8584a2ccd909937f. Latest consensus state: current_term: 1 leader_uuid: "a6b163d5d844464b8584a2ccd909937f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b163d5d844464b8584a2ccd909937f" member_type: VOTER } }
I20260811 01:57:44.484136 16190 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.484328 16194 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:44.485498 16194 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:44.485672 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:44.487524 16194 catalog_manager.cc:1383] Generated new cluster ID: 7daf9bd34d2b4bdb81b32fe0ee42084a
I20260811 01:57:44.487576 16194 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:44.495959 16194 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:44.496483 16194 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:44.504386 16194 catalog_manager.cc:6092] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f: Generated new TSK 0
I20260811 01:57:44.504560 16194 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:44.517892 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.519867 16218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.519999 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.519894 16217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.520068 16221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.520287 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.520334 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.520354 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464520354 us; error 0 us; skew 500 ppm
I20260811 01:57:44.521168 10906 webserver.cc:533] Webserver started at http://127.10.166.129:33421/ using document root <none> and password file <none>
I20260811 01:57:44.521334 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.521391 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.521474 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.521886 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "96b7a542100e4695a17bd2eb20312ba8"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.523310 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:44.524221 16227 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.524441 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:44.524508 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "96b7a542100e4695a17bd2eb20312ba8"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.524574 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.528931 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.529218 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.529933 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:44.529984 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.530025 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:44.530050 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.534984 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:37537
I20260811 01:57:44.536046 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260811 01:57:44.536617 16330 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:37537 every 8 connection(s)
I20260811 01:57:44.537472 16338 heartbeater.cc:344] Connected to a master server at 127.10.166.190:43047
I20260811 01:57:44.537611 16338 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.537849 16338 heartbeater.cc:507] Master 127.10.166.190:43047 requested a full tablet report, sending...
W20260811 01:57:44.538113 16348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.538116 16347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.538239 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:44.538456 16116 ts_manager.cc:194] Registered new tserver with Master: 96b7a542100e4695a17bd2eb20312ba8 (127.10.166.129:37537)
I20260811 01:57:44.539198 16116 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54010
W20260811 01:57:44.539816 16355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.540102 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.540162 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.540184 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464540183 us; error 0 us; skew 500 ppm
I20260811 01:57:44.541023 10906 webserver.cc:533] Webserver started at http://127.10.166.130:35115/ using document root <none> and password file <none>
I20260811 01:57:44.541172 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.541221 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.541286 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.541725 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "d3dd498ceb8a44d59986059fd6f26815"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.543205 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.544130 16367 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.544410 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:44.544471 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "d3dd498ceb8a44d59986059fd6f26815"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.544536 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.556955 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.557217 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.557885 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:44.557950 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.557999 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:44.558029 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.563462 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:40675
I20260811 01:57:44.563486 16496 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:40675 every 8 connection(s)
I20260811 01:57:44.564392 16497 heartbeater.cc:344] Connected to a master server at 127.10.166.190:43047
I20260811 01:57:44.564466 16497 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.564642 16497 heartbeater.cc:507] Master 127.10.166.190:43047 requested a full tablet report, sending...
I20260811 01:57:44.565191 16116 ts_manager.cc:194] Registered new tserver with Master: d3dd498ceb8a44d59986059fd6f26815 (127.10.166.130:40675)
I20260811 01:57:44.565729 16116 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54022
I20260811 01:57:44.566184 10906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.002304747s
I20260811 01:57:44.575052 16268 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 01:57:44.575352 16268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.577246 16513 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Bootstrap starting.
I20260811 01:57:44.578150 16513 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.579167 16513 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: No bootstrap required, opened a new log
I20260811 01:57:44.579241 16513 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:44.579587 16513 raft_consensus.cc:359] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.579666 16513 raft_consensus.cc:385] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.579692 16513 raft_consensus.cc:740] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b7a542100e4695a17bd2eb20312ba8, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.579836 16513 consensus_queue.cc:260] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.579900 16513 raft_consensus.cc:399] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.579933 16513 raft_consensus.cc:493] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.579972 16513 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.580631 16513 raft_consensus.cc:515] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.580746 16513 leader_election.cc:304] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b7a542100e4695a17bd2eb20312ba8; no voters:
I20260811 01:57:44.580894 16513 leader_election.cc:290] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.581001 16517 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.581189 16513 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:44.581230 16338 heartbeater.cc:499] Master 127.10.166.190:43047 was elected leader, sending a full tablet report...
I20260811 01:57:44.581250 16517 raft_consensus.cc:697] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 96b7a542100e4695a17bd2eb20312ba8, State: Running, Role: LEADER
I20260811 01:57:44.581414 16517 consensus_queue.cc:237] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.581809 16519 tablet_replica.cc:442] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b7a542100e4695a17bd2eb20312ba8" committed_config { opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } } }
I20260811 01:57:44.581897 16519 tablet_replica.cc:445] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.581907 16521 tablet_replica.cc:442] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: TxnStatusTablet state changed. Reason: New leader 96b7a542100e4695a17bd2eb20312ba8. Latest consensus state: current_term: 1 leader_uuid: "96b7a542100e4695a17bd2eb20312ba8" committed_config { opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } } }
I20260811 01:57:44.582016 16521 tablet_replica.cc:445] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.582177 16523 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:57:44.582266 16523 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260811 01:57:44.582494 16116 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260811 01:57:44.583535 16116 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.595575 16530 ts_tablet_manager.cc:933] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Initiating tablet copy from peer 96b7a542100e4695a17bd2eb20312ba8 (127.10.166.129:37537)
I20260811 01:57:44.595875 16530 tablet_copy_client.cc:323] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: tablet copy: Beginning tablet copy session from remote peer at address 127.10.166.129:37537
I20260811 01:57:44.598589 16304 tablet_copy_service.cc:140] P 96b7a542100e4695a17bd2eb20312ba8: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer d3dd498ceb8a44d59986059fd6f26815 ({username='slave'} at 127.0.0.1:34896)
I20260811 01:57:44.598680 16304 tablet_copy_service.cc:161] P 96b7a542100e4695a17bd2eb20312ba8: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer d3dd498ceb8a44d59986059fd6f26815 at {username='slave'} at 127.0.0.1:34896: session id = d3dd498ceb8a44d59986059fd6f26815-11111111111111111111111111111111
I20260811 01:57:44.599382 16304 tablet_copy_source_session.cc:215] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:57:44.600142 16530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.601794 16530 tablet_copy_client.cc:806] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: tablet copy: Starting download of 0 data blocks...
I20260811 01:57:44.601948 16530 tablet_copy_client.cc:670] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: tablet copy: Starting download of 1 WAL segments...
I20260811 01:57:44.602944 16530 tablet_copy_client.cc:538] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:57:44.603974 16530 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Bootstrap starting.
I20260811 01:57:44.608654 16530 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:44.608865 16530 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Bootstrap complete.
I20260811 01:57:44.609017 16530 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 01:57:44.609433 16530 raft_consensus.cc:359] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.609519 16530 raft_consensus.cc:740] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: d3dd498ceb8a44d59986059fd6f26815, State: Initialized, Role: NON_PARTICIPANT
I20260811 01:57:44.609603 16530 consensus_queue.cc:260] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.609886 16530 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.609900 16537 tablet_replica.cc:442] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } } }
I20260811 01:57:44.609966 16497 heartbeater.cc:499] Master 127.10.166.190:43047 was elected leader, sending a full tablet report...
I20260811 01:57:44.609989 16537 tablet_replica.cc:445] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260811 01:57:44.610334 16304 tablet_copy_service.cc:342] P 96b7a542100e4695a17bd2eb20312ba8: Request end of tablet copy session d3dd498ceb8a44d59986059fd6f26815-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:34896
I20260811 01:57:44.610397 16304 tablet_copy_service.cc:434] P 96b7a542100e4695a17bd2eb20312ba8: ending tablet copy session d3dd498ceb8a44d59986059fd6f26815-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer d3dd498ceb8a44d59986059fd6f26815
W20260811 01:57:44.610643 16116 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.617508 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:44.620520 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.620754 10906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: stopping tablet replica
I20260811 01:57:44.620862 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.620978 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.633863 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:44.636266 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.637777 16538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.637892 16539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.637943 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.637998 16546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.638227 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.638273 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.638288 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464638288 us; error 0 us; skew 500 ppm
I20260811 01:57:44.639072 10906 webserver.cc:533] Webserver started at http://127.10.166.129:33421/ using document root <none> and password file <none>
I20260811 01:57:44.639216 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.639250 10906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260811 01:57:44.640235 10906 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.640997 16554 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.641199 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.641259 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "96b7a542100e4695a17bd2eb20312ba8"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.641333 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.647565 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.647873 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.649672 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 01:57:44.649731 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:44.650565 10906 ts_tablet_manager.cc:616] Registered 1 tablets
I20260811 01:57:44.650606 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:44.650684 16564 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Bootstrap starting.
I20260811 01:57:44.656021 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:37537
I20260811 01:57:44.656046 16659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:37537 every 8 connection(s)
I20260811 01:57:44.656458 16564 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:44.656766 16564 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Bootstrap complete.
I20260811 01:57:44.656962 16564 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 01:57:44.657367 16660 heartbeater.cc:344] Connected to a master server at 127.10.166.190:43047
I20260811 01:57:44.657454 16660 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.657415 16564 raft_consensus.cc:359] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.657510 16564 raft_consensus.cc:740] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b7a542100e4695a17bd2eb20312ba8, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.657622 16564 consensus_queue.cc:260] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.657662 16660 heartbeater.cc:507] Master 127.10.166.190:43047 requested a full tablet report, sending...
I20260811 01:57:44.657706 16564 raft_consensus.cc:399] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.657761 16564 raft_consensus.cc:493] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.657810 16564 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:44.658396 16116 ts_manager.cc:194] Re-registered known tserver with Master: 96b7a542100e4695a17bd2eb20312ba8 (127.10.166.129:37537)
W20260811 01:57:44.658496 16116 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.658737 16564 raft_consensus.cc:515] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.658867 16564 leader_election.cc:304] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b7a542100e4695a17bd2eb20312ba8; no voters:
I20260811 01:57:44.658896 16116 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54060
I20260811 01:57:44.659055 16564 leader_election.cc:290] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:57:44.659166 16666 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:44.659354 16564 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 01:57:44.659658 16660 heartbeater.cc:499] Master 127.10.166.190:43047 was elected leader, sending a full tablet report...
I20260811 01:57:44.659536 16666 raft_consensus.cc:697] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 2 LEADER]: Becoming Leader. State: Replica: 96b7a542100e4695a17bd2eb20312ba8, State: Running, Role: LEADER
I20260811 01:57:44.659880 16666 consensus_queue.cc:237] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } }
I20260811 01:57:44.660254 16667 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96b7a542100e4695a17bd2eb20312ba8" committed_config { opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } } }
I20260811 01:57:44.660336 16667 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.660353 16668 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader 96b7a542100e4695a17bd2eb20312ba8. Latest consensus state: current_term: 2 leader_uuid: "96b7a542100e4695a17bd2eb20312ba8" committed_config { opid_index: -1 peers { permanent_uuid: "96b7a542100e4695a17bd2eb20312ba8" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 37537 } } }
I20260811 01:57:44.660429 16668 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.660877 16671 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
W20260811 01:57:44.660938 16116 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.660948 16671 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260811 01:57:44.664728 10906 tablet_server.cc:179] TabletServer@127.10.166.129:37537 shutting down...
I20260811 01:57:44.667677 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.667904 10906 tablet_replica.cc:333] stopping tablet replica
I20260811 01:57:44.668035 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 2 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.668133 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 96b7a542100e4695a17bd2eb20312ba8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.670277 10906 tablet_server.cc:196] TabletServer@127.10.166.129:37537 shutdown complete.
I20260811 01:57:44.672305 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:44.675439 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.675630 10906 tablet_replica.cc:333] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815: stopping tablet replica
I20260811 01:57:44.675752 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 01:57:44.675849 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P d3dd498ceb8a44d59986059fd6f26815 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 01:57:44.687943 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:44.689909 10906 master.cc:562] Master@127.10.166.190:43047 shutting down...
I20260811 01:57:44.692725 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.692840 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.692939 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P a6b163d5d844464b8584a2ccd909937f: stopping tablet replica
I20260811 01:57:44.705090 10906 master.cc:584] Master@127.10.166.190:43047 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (266 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260811 01:57:44.710521 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:40797
I20260811 01:57:44.710834 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.712244 16681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.712311 16679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.712400 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.712545 16683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.712740 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.712781 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.712795 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464712794 us; error 0 us; skew 500 ppm
I20260811 01:57:44.713498 10906 webserver.cc:533] Webserver started at http://127.10.166.190:40955/ using document root <none> and password file <none>
I20260811 01:57:44.713634 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.713675 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.713740 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.714078 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "93e8b5799db145aba1da2a1419d6ebde"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.715428 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.716285 16689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.716498 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 01:57:44.716558 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "93e8b5799db145aba1da2a1419d6ebde"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.716624 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.728094 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.728349 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.731922 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:40797
I20260811 01:57:44.733510 16779 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:40797 every 8 connection(s)
I20260811 01:57:44.733886 16780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.735512 16780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde: Bootstrap starting.
I20260811 01:57:44.736270 16780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.737143 16780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde: No bootstrap required, opened a new log
I20260811 01:57:44.737489 16780 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER }
I20260811 01:57:44.737566 16780 raft_consensus.cc:385] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.737591 16780 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e8b5799db145aba1da2a1419d6ebde, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.737685 16780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER }
I20260811 01:57:44.737736 16780 raft_consensus.cc:399] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.737762 16780 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.737794 16780 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.738387 16780 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER }
I20260811 01:57:44.738504 16780 leader_election.cc:304] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93e8b5799db145aba1da2a1419d6ebde; no voters:
I20260811 01:57:44.738651 16780 leader_election.cc:290] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.738773 16785 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.738940 16785 raft_consensus.cc:697] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 1 LEADER]: Becoming Leader. State: Replica: 93e8b5799db145aba1da2a1419d6ebde, State: Running, Role: LEADER
I20260811 01:57:44.739099 16780 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:44.739063 16785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER }
I20260811 01:57:44.739507 16786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93e8b5799db145aba1da2a1419d6ebde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER } }
I20260811 01:57:44.739533 16787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93e8b5799db145aba1da2a1419d6ebde. Latest consensus state: current_term: 1 leader_uuid: "93e8b5799db145aba1da2a1419d6ebde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e8b5799db145aba1da2a1419d6ebde" member_type: VOTER } }
I20260811 01:57:44.739727 16787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.739670 16786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.740324 16793 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:44.741175 16793 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:44.741369 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:44.742887 16793 catalog_manager.cc:1383] Generated new cluster ID: a7ad7aee06e14d409a385ac6924e3cf8
I20260811 01:57:44.742942 16793 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:44.748777 16793 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:44.749481 16793 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:44.756173 16793 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde: Generated new TSK 0
I20260811 01:57:44.756330 16793 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:44.757444 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.759446 16822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.759596 16816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.759615 16819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.759618 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:44.759992 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.760046 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.760069 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464760068 us; error 0 us; skew 500 ppm
I20260811 01:57:44.761054 10906 webserver.cc:533] Webserver started at http://127.10.166.129:43707/ using document root <none> and password file <none>
I20260811 01:57:44.761240 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.761301 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.761389 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.761847 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "1209e1c4ce0e491998ece508c98cebeb"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.763581 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.764780 16829 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.765060 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.765136 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "1209e1c4ce0e491998ece508c98cebeb"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.765218 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.775352 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.775837 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.776688 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:44.776747 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.776793 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:44.776818 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.781934 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:33443
I20260811 01:57:44.781968 16943 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:33443 every 8 connection(s)
I20260811 01:57:44.783013 16950 heartbeater.cc:344] Connected to a master server at 127.10.166.190:40797
I20260811 01:57:44.783113 16950 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.783358 16950 heartbeater.cc:507] Master 127.10.166.190:40797 requested a full tablet report, sending...
I20260811 01:57:44.784037 16711 ts_manager.cc:194] Registered new tserver with Master: 1209e1c4ce0e491998ece508c98cebeb (127.10.166.129:33443)
I20260811 01:57:44.784644 10906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002217939s
I20260811 01:57:44.784987 16711 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43682
I20260811 01:57:44.794364 16884 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 01:57:44.794687 16884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.796708 16968 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: Bootstrap starting.
I20260811 01:57:44.797580 16968 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.798503 16968 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: No bootstrap required, opened a new log
I20260811 01:57:44.798578 16968 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:44.798921 16968 raft_consensus.cc:359] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } }
I20260811 01:57:44.799011 16968 raft_consensus.cc:385] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.799043 16968 raft_consensus.cc:740] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1209e1c4ce0e491998ece508c98cebeb, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.799149 16968 consensus_queue.cc:260] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } }
I20260811 01:57:44.799203 16968 raft_consensus.cc:399] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.799229 16968 raft_consensus.cc:493] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.799263 16968 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.800021 16968 raft_consensus.cc:515] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } }
I20260811 01:57:44.800144 16968 leader_election.cc:304] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1209e1c4ce0e491998ece508c98cebeb; no voters:
I20260811 01:57:44.800303 16968 leader_election.cc:290] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.800405 16971 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.800621 16968 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 01:57:44.800624 16950 heartbeater.cc:499] Master 127.10.166.190:40797 was elected leader, sending a full tablet report...
I20260811 01:57:44.800630 16971 raft_consensus.cc:697] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 1 LEADER]: Becoming Leader. State: Replica: 1209e1c4ce0e491998ece508c98cebeb, State: Running, Role: LEADER
I20260811 01:57:44.800822 16971 consensus_queue.cc:237] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } }
I20260811 01:57:44.801256 16974 tablet_replica.cc:442] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1209e1c4ce0e491998ece508c98cebeb" committed_config { opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } } }
I20260811 01:57:44.801281 16975 tablet_replica.cc:442] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: TxnStatusTablet state changed. Reason: New leader 1209e1c4ce0e491998ece508c98cebeb. Latest consensus state: current_term: 1 leader_uuid: "1209e1c4ce0e491998ece508c98cebeb" committed_config { opid_index: -1 peers { permanent_uuid: "1209e1c4ce0e491998ece508c98cebeb" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 33443 } } }
I20260811 01:57:44.801347 16974 tablet_replica.cc:445] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.801360 16975 tablet_replica.cc:445] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.801566 16977 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:57:44.801657 16977 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20260811 01:57:44.801994 16711 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260811 01:57:44.814803 16944 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 01:57:44.814878 16944 txn_status_manager.cc:244] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 01:57:44.814951 16944 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260811 01:57:44.816728 16944 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 01:57:44.825670 16944 txn_status_manager.cc:244] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260811 01:57:44.825721 16944 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260811 01:57:44.827454 16944 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260811 01:57:44.830753 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.831379 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.831946 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.832481 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.833012 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.833586 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:38498
W20260811 01:57:44.836555 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
W20260811 01:57:44.837105 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
W20260811 01:57:44.837651 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
W20260811 01:57:44.838189 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
W20260811 01:57:44.838722 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
W20260811 01:57:44.839254 16884 server_base.cc:1143] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:38514
I20260811 01:57:44.839949 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:44.843233 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.843437 10906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb: stopping tablet replica
I20260811 01:57:44.843544 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.843657 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 1209e1c4ce0e491998ece508c98cebeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.856577 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:44.858677 10906 master.cc:562] Master@127.10.166.190:40797 shutting down...
I20260811 01:57:44.861701 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.861819 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.861919 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 93e8b5799db145aba1da2a1419d6ebde: stopping tablet replica
I20260811 01:57:44.864041 10906 master.cc:584] Master@127.10.166.190:40797 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (158 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260811 01:57:44.869038 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:34629
I20260811 01:57:44.869354 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.870685 16993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.870740 16996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.870766 16999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.870899 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:44.871029 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.871066 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.871085 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464871085 us; error 0 us; skew 500 ppm
I20260811 01:57:44.871826 10906 webserver.cc:533] Webserver started at http://127.10.166.190:39335/ using document root <none> and password file <none>
I20260811 01:57:44.871970 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.872014 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.872080 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.872409 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "c93c9e3996e44dc58e79e37b599e6b11"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.873706 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.874500 17004 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.874706 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.874773 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "c93c9e3996e44dc58e79e37b599e6b11"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.874837 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.896844 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.897135 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.900704 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:34629
I20260811 01:57:44.901633 17100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:34629 every 8 connection(s)
I20260811 01:57:44.902037 17101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.903632 17101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11: Bootstrap starting.
I20260811 01:57:44.904388 17101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.905289 17101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11: No bootstrap required, opened a new log
I20260811 01:57:44.905663 17101 raft_consensus.cc:359] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER }
I20260811 01:57:44.905740 17101 raft_consensus.cc:385] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.905771 17101 raft_consensus.cc:740] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93c9e3996e44dc58e79e37b599e6b11, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.905900 17101 consensus_queue.cc:260] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER }
I20260811 01:57:44.905967 17101 raft_consensus.cc:399] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.906004 17101 raft_consensus.cc:493] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.906050 17101 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.906694 17101 raft_consensus.cc:515] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER }
I20260811 01:57:44.906810 17101 leader_election.cc:304] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c93c9e3996e44dc58e79e37b599e6b11; no voters:
I20260811 01:57:44.906983 17101 leader_election.cc:290] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.907071 17110 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.907249 17110 raft_consensus.cc:697] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 1 LEADER]: Becoming Leader. State: Replica: c93c9e3996e44dc58e79e37b599e6b11, State: Running, Role: LEADER
I20260811 01:57:44.907387 17101 sys_catalog.cc:565] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:44.907384 17110 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER }
I20260811 01:57:44.907866 17111 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c93c9e3996e44dc58e79e37b599e6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER } }
I20260811 01:57:44.907894 17112 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c93c9e3996e44dc58e79e37b599e6b11. Latest consensus state: current_term: 1 leader_uuid: "c93c9e3996e44dc58e79e37b599e6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93c9e3996e44dc58e79e37b599e6b11" member_type: VOTER } }
I20260811 01:57:44.908059 17112 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.908034 17111 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:44.908466 17119 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:44.909221 17119 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:44.909408 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:44.910935 17119 catalog_manager.cc:1383] Generated new cluster ID: 9d34b264671c4605b309a05b2b800f03
I20260811 01:57:44.910989 17119 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:44.917742 17119 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:44.918201 17119 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:44.922117 17119 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11: Generated new TSK 0
I20260811 01:57:44.922240 17119 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:44.925232 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:44.926736 17142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:44.926767 17136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.926806 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:44.926909 17140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:44.927094 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:44.927134 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:44.927147 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413464927147 us; error 0 us; skew 500 ppm
I20260811 01:57:44.927896 10906 webserver.cc:533] Webserver started at http://127.10.166.129:38949/ using document root <none> and password file <none>
I20260811 01:57:44.928015 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:44.928050 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:44.928098 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:44.928380 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "8aedb8f0481e47138cf88cc8c1b09df6"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.929628 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:44.930408 17150 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.930612 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:44.930670 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "8aedb8f0481e47138cf88cc8c1b09df6"
format_stamp: "Formatted at 2026-08-11 01:57:44 on dist-test-slave-1l3l"
I20260811 01:57:44.930734 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:44.946122 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:44.946429 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:44.947165 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:44.947208 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.947252 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:44.947288 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:44.952531 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:34535
I20260811 01:57:44.952564 17268 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:34535 every 8 connection(s)
I20260811 01:57:44.953696 17273 heartbeater.cc:344] Connected to a master server at 127.10.166.190:34629
I20260811 01:57:44.953785 17273 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:44.953975 17273 heartbeater.cc:507] Master 127.10.166.190:34629 requested a full tablet report, sending...
I20260811 01:57:44.954527 17041 ts_manager.cc:194] Registered new tserver with Master: 8aedb8f0481e47138cf88cc8c1b09df6 (127.10.166.129:34535)
I20260811 01:57:44.955204 10906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002228492s
I20260811 01:57:44.955488 17041 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38000
I20260811 01:57:44.964907 17204 tablet_service.cc:1511] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 01:57:44.965193 17204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.967094 17293 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: Bootstrap starting.
I20260811 01:57:44.967996 17293 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.968998 17293 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: No bootstrap required, opened a new log
I20260811 01:57:44.969089 17293 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:44.969501 17293 raft_consensus.cc:359] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.969588 17293 raft_consensus.cc:385] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.969619 17293 raft_consensus.cc:740] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aedb8f0481e47138cf88cc8c1b09df6, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.969748 17293 consensus_queue.cc:260] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.969817 17293 raft_consensus.cc:399] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.969856 17293 raft_consensus.cc:493] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.969902 17293 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.970662 17293 raft_consensus.cc:515] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.970790 17293 leader_election.cc:304] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aedb8f0481e47138cf88cc8c1b09df6; no voters:
I20260811 01:57:44.970966 17293 leader_election.cc:290] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.971060 17297 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.971230 17297 raft_consensus.cc:697] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 LEADER]: Becoming Leader. State: Replica: 8aedb8f0481e47138cf88cc8c1b09df6, State: Running, Role: LEADER
I20260811 01:57:44.971256 17293 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:44.971403 17273 heartbeater.cc:499] Master 127.10.166.190:34629 was elected leader, sending a full tablet report...
I20260811 01:57:44.971365 17297 consensus_queue.cc:237] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.971808 17300 tablet_replica.cc:442] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: TxnStatusTablet state changed. Reason: New leader 8aedb8f0481e47138cf88cc8c1b09df6. Latest consensus state: current_term: 1 leader_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" committed_config { opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } } }
I20260811 01:57:44.971889 17300 tablet_replica.cc:445] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.971897 17299 tablet_replica.cc:442] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" committed_config { opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } } }
I20260811 01:57:44.972020 17299 tablet_replica.cc:445] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:57:44.972445 17304 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
W20260811 01:57:44.972455 17041 catalog_manager.cc:5552] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260811 01:57:44.972545 17304 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20260811 01:57:44.979562 17204 tablet_service.cc:1511] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260811 01:57:44.979847 17204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:44.981423 17293 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: Bootstrap starting.
I20260811 01:57:44.982345 17293 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:44.983301 17293 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: No bootstrap required, opened a new log
I20260811 01:57:44.983374 17293 ts_tablet_manager.cc:1403] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:57:44.983820 17293 raft_consensus.cc:359] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.983898 17293 raft_consensus.cc:385] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:44.983930 17293 raft_consensus.cc:740] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aedb8f0481e47138cf88cc8c1b09df6, State: Initialized, Role: FOLLOWER
I20260811 01:57:44.984054 17293 consensus_queue.cc:260] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.984125 17293 raft_consensus.cc:399] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:44.984161 17293 raft_consensus.cc:493] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:44.984207 17293 raft_consensus.cc:3060] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:44.984884 17293 raft_consensus.cc:515] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.985000 17293 leader_election.cc:304] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aedb8f0481e47138cf88cc8c1b09df6; no voters:
I20260811 01:57:44.985100 17293 leader_election.cc:290] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:44.985160 17300 raft_consensus.cc:2804] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:44.985265 17300 raft_consensus.cc:697] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 LEADER]: Becoming Leader. State: Replica: 8aedb8f0481e47138cf88cc8c1b09df6, State: Running, Role: LEADER
I20260811 01:57:44.985381 17300 consensus_queue.cc:237] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "8aedb8f0481e47138cf88cc8c1b09df6" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 34535 } }
I20260811 01:57:44.985528 17293 ts_tablet_manager.cc:1434] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 01:57:44.986400 17041 catalog_manager.cc:5552] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260811 01:57:44.987432 17041 catalog_manager.cc:5552] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260811 01:57:44.992976 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:44.996100 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:44.996296 10906 tablet_replica.cc:333] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6: stopping tablet replica
I20260811 01:57:44.996398 10906 raft_consensus.cc:2243] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.996503 10906 raft_consensus.cc:2272] T 22222222222222222222222222222222 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:44.997030 10906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6: stopping tablet replica
I20260811 01:57:44.997104 10906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:44.997200 10906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 8aedb8f0481e47138cf88cc8c1b09df6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:45.009927 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:45.012322 10906 master.cc:562] Master@127.10.166.190:34629 shutting down...
I20260811 01:57:45.015048 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:45.015185 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:45.015295 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P c93c9e3996e44dc58e79e37b599e6b11: stopping tablet replica
I20260811 01:57:45.027793 10906 master.cc:584] Master@127.10.166.190:34629 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (164 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (819 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260811 01:57:45.033138 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:32845
I20260811 01:57:45.033444 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:45.034863 17321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.034874 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:45.034818 17327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.035001 17330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.035171 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:45.035211 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:45.035230 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413465035230 us; error 0 us; skew 500 ppm
I20260811 01:57:45.036013 10906 webserver.cc:533] Webserver started at http://127.10.166.190:41727/ using document root <none> and password file <none>
I20260811 01:57:45.036160 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:45.036209 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:45.036278 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:45.036628 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "0e9aba7ccdc44134a3fd532a5f8c7663"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.037961 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:45.038765 17340 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.038971 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.039034 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "0e9aba7ccdc44134a3fd532a5f8c7663"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.039099 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:45.044812 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:45.045076 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:45.048694 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:32845
I20260811 01:57:45.049533 17435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:32845 every 8 connection(s)
I20260811 01:57:45.049913 17436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:45.051481 17436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663: Bootstrap starting.
I20260811 01:57:45.052363 17436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:45.053265 17436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663: No bootstrap required, opened a new log
I20260811 01:57:45.053651 17436 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER }
I20260811 01:57:45.053733 17436 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:45.053766 17436 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9aba7ccdc44134a3fd532a5f8c7663, State: Initialized, Role: FOLLOWER
I20260811 01:57:45.053889 17436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER }
I20260811 01:57:45.053956 17436 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:45.053995 17436 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:45.054041 17436 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:45.054701 17436 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER }
I20260811 01:57:45.054833 17436 leader_election.cc:304] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e9aba7ccdc44134a3fd532a5f8c7663; no voters:
I20260811 01:57:45.054996 17436 leader_election.cc:290] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:45.055081 17440 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:45.055321 17440 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9aba7ccdc44134a3fd532a5f8c7663, State: Running, Role: LEADER
I20260811 01:57:45.055444 17436 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:45.055542 17440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER }
I20260811 01:57:45.055980 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER } }
I20260811 01:57:45.056006 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e9aba7ccdc44134a3fd532a5f8c7663. Latest consensus state: current_term: 1 leader_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9aba7ccdc44134a3fd532a5f8c7663" member_type: VOTER } }
I20260811 01:57:45.056124 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:45.056154 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:45.056792 17453 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:45.057531 17453 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:45.057714 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:45.059247 17453 catalog_manager.cc:1383] Generated new cluster ID: ea0aff2d34584c209061cd169ac4ace0
I20260811 01:57:45.059300 17453 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:45.065985 17453 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:45.066448 17453 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:45.073658 17453 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663: Generated new TSK 0
I20260811 01:57:45.073786 17453 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:45.089834 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:45.091405 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.091411 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.091614 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:45.091440 17472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.091857 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:45.091902 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:45.091915 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413465091915 us; error 0 us; skew 500 ppm
I20260811 01:57:45.092695 10906 webserver.cc:533] Webserver started at http://127.10.166.129:36811/ using document root <none> and password file <none>
I20260811 01:57:45.092844 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:45.092890 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:45.092965 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:45.093362 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "ce8f83ac34584178a56c1937d383b73e"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.094723 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:45.095551 17482 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.095780 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.095846 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "ce8f83ac34584178a56c1937d383b73e"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.095912 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:45.102435 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:45.102703 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:45.102948 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:45.103358 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:45.103394 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.103431 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:45.103459 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.107231 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:38507
I20260811 01:57:45.107295 17578 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:38507 every 8 connection(s)
I20260811 01:57:45.108021 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:45.109481 17590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.109481 17585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.109575 17583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.109783 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:45.109927 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:45.109974 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:45.109994 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413465109994 us; error 0 us; skew 500 ppm
I20260811 01:57:45.110791 10906 webserver.cc:533] Webserver started at http://127.10.166.130:42369/ using document root <none> and password file <none>
I20260811 01:57:45.110952 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:45.111007 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:45.111070 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:45.111445 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "d6cd6377ae804351a400f085ec7bfb33"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.112365 17579 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32845
I20260811 01:57:45.112463 17579 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:45.112663 17579 heartbeater.cc:507] Master 127.10.166.190:32845 requested a full tablet report, sending...
I20260811 01:57:45.113169 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:45.113260 17367 ts_manager.cc:194] Registered new tserver with Master: ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507)
I20260811 01:57:45.113938 17367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58738
I20260811 01:57:45.114107 17602 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.114435 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.114502 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "d6cd6377ae804351a400f085ec7bfb33"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.114567 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:45.120971 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:45.121230 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:45.121475 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:45.121855 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:45.121891 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.121930 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:45.121959 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.126221 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:33351
I20260811 01:57:45.126243 17709 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:33351 every 8 connection(s)
I20260811 01:57:45.126902 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:45.128391 17715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.128499 17716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:45.128672 17719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:45.129123 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:45.129268 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:45.129307 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:45.129328 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413465129327 us; error 0 us; skew 500 ppm
I20260811 01:57:45.130117 10906 webserver.cc:533] Webserver started at http://127.10.166.131:45725/ using document root <none> and password file <none>
I20260811 01:57:45.130255 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:45.130301 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:45.130367 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:45.130402 17711 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32845
I20260811 01:57:45.130488 17711 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:45.130672 17711 heartbeater.cc:507] Master 127.10.166.190:32845 requested a full tablet report, sending...
I20260811 01:57:45.130788 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root/instance:
uuid: "24da0820c3c14d0f8507938315d91e91"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.131232 17367 ts_manager.cc:194] Registered new tserver with Master: d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351)
I20260811 01:57:45.131861 17367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58750
I20260811 01:57:45.132426 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:45.133318 17728 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.133512 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.133575 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root
uuid: "24da0820c3c14d0f8507938315d91e91"
format_stamp: "Formatted at 2026-08-11 01:57:45 on dist-test-slave-1l3l"
I20260811 01:57:45.133638 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:45.140269 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:45.140633 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:45.140923 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:45.141350 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:45.141390 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.141433 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:45.141460 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:45.146281 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.131:34415
I20260811 01:57:45.146335 17833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.131:34415 every 8 connection(s)
I20260811 01:57:45.150089 17834 heartbeater.cc:344] Connected to a master server at 127.10.166.190:32845
I20260811 01:57:45.150174 17834 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:45.150348 17834 heartbeater.cc:507] Master 127.10.166.190:32845 requested a full tablet report, sending...
I20260811 01:57:45.150861 17367 ts_manager.cc:194] Registered new tserver with Master: 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:45.151113 10906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004464142s
I20260811 01:57:45.151459 17367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58762
I20260811 01:57:45.163286 17367 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58786:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:57:45.172435 17652 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 (DEFAULT_TABLE table=test-workload [id=e65ac1d8cb0645f389cdf77339884cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:45.172746 17652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3aabcc3e274eebb1c1e30edd9a0845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:45.172706 17522 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 (DEFAULT_TABLE table=test-workload [id=e65ac1d8cb0645f389cdf77339884cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:45.172979 17522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3aabcc3e274eebb1c1e30edd9a0845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:45.173480 17773 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 (DEFAULT_TABLE table=test-workload [id=e65ac1d8cb0645f389cdf77339884cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:45.173727 17773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3aabcc3e274eebb1c1e30edd9a0845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:45.175046 17871 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: Bootstrap starting.
I20260811 01:57:45.175733 17873 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Bootstrap starting.
I20260811 01:57:45.176148 17871 tablet_bootstrap.cc:654] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:45.176338 17872 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Bootstrap starting.
I20260811 01:57:45.176831 17873 tablet_bootstrap.cc:654] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:45.177260 17871 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: No bootstrap required, opened a new log
I20260811 01:57:45.177299 17872 tablet_bootstrap.cc:654] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:45.177362 17871 ts_tablet_manager.cc:1403] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:45.177747 17871 raft_consensus.cc:359] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.177836 17871 raft_consensus.cc:385] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:45.177868 17871 raft_consensus.cc:740] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6cd6377ae804351a400f085ec7bfb33, State: Initialized, Role: FOLLOWER
I20260811 01:57:45.177997 17871 consensus_queue.cc:260] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.178354 17873 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: No bootstrap required, opened a new log
I20260811 01:57:45.178385 17711 heartbeater.cc:499] Master 127.10.166.190:32845 was elected leader, sending a full tablet report...
I20260811 01:57:45.178431 17872 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: No bootstrap required, opened a new log
I20260811 01:57:45.178443 17873 ts_tablet_manager.cc:1403] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:57:45.178527 17872 ts_tablet_manager.cc:1403] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:45.178750 17871 ts_tablet_manager.cc:1434] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.178996 17872 raft_consensus.cc:359] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.179088 17872 raft_consensus.cc:385] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:45.179051 17873 raft_consensus.cc:359] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.179119 17872 raft_consensus.cc:740] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8f83ac34584178a56c1937d383b73e, State: Initialized, Role: FOLLOWER
I20260811 01:57:45.179143 17873 raft_consensus.cc:385] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:45.179177 17873 raft_consensus.cc:740] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24da0820c3c14d0f8507938315d91e91, State: Initialized, Role: FOLLOWER
I20260811 01:57:45.179247 17872 consensus_queue.cc:260] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.179291 17873 consensus_queue.cc:260] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.179589 17872 ts_tablet_manager.cc:1434] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.179627 17873 ts_tablet_manager.cc:1434] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:45.179759 17579 heartbeater.cc:499] Master 127.10.166.190:32845 was elected leader, sending a full tablet report...
I20260811 01:57:45.180027 17834 heartbeater.cc:499] Master 127.10.166.190:32845 was elected leader, sending a full tablet report...
I20260811 01:57:45.191951 17881 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:45.192034 17881 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.192480 17881 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507)
I20260811 01:57:45.196089 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "24da0820c3c14d0f8507938315d91e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33" is_pre_election: true
I20260811 01:57:45.196089 17539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "24da0820c3c14d0f8507938315d91e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8f83ac34584178a56c1937d383b73e" is_pre_election: true
I20260811 01:57:45.196247 17539 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24da0820c3c14d0f8507938315d91e91 in term 0.
I20260811 01:57:45.196247 17666 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24da0820c3c14d0f8507938315d91e91 in term 0.
I20260811 01:57:45.196583 17732 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24da0820c3c14d0f8507938315d91e91, ce8f83ac34584178a56c1937d383b73e; no voters:
I20260811 01:57:45.196762 17881 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:45.196825 17881 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:45.196871 17881 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:45.197693 17881 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.198093 17881 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [CANDIDATE]: Term 1 election: Requested vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507)
I20260811 01:57:45.198380 17539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "24da0820c3c14d0f8507938315d91e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8f83ac34584178a56c1937d383b73e"
I20260811 01:57:45.198380 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "24da0820c3c14d0f8507938315d91e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33"
I20260811 01:57:45.198519 17666 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:45.198519 17539 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:45.199503 17666 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24da0820c3c14d0f8507938315d91e91 in term 1.
I20260811 01:57:45.199503 17539 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24da0820c3c14d0f8507938315d91e91 in term 1.
I20260811 01:57:45.199865 17730 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24da0820c3c14d0f8507938315d91e91, ce8f83ac34584178a56c1937d383b73e, d6cd6377ae804351a400f085ec7bfb33; no voters:
I20260811 01:57:45.200050 17881 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:45.200235 17881 raft_consensus.cc:697] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 1 LEADER]: Becoming Leader. State: Replica: 24da0820c3c14d0f8507938315d91e91, State: Running, Role: LEADER
I20260811 01:57:45.200373 17881 consensus_queue.cc:237] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:45.201925 17367 catalog_manager.cc:5719] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24da0820c3c14d0f8507938315d91e91 (127.10.166.131). New cstate: current_term: 1 leader_uuid: "24da0820c3c14d0f8507938315d91e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } } }
I20260811 01:57:45.256800 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.256810 17773 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:45.257823 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 1 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): RUNNING [LEADER]
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:45.282912 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:45.474537 17773 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:45.474553 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.474759 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 1 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): RUNNING [LEADER]
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:45.497478 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:45.624847 17881 consensus_queue.cc:1048] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:45.627903 17881 consensus_queue.cc:1048] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:45.770897 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.770890 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.771028 17773 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 1 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 01:57:45.800921 10906 log_verifier.cc:126] Checking tablet 1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:45.823839 10906 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:45.824048 10906 tablet_replica.cc:333] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: stopping tablet replica
I20260811 01:57:45.824162 10906 raft_consensus.cc:2243] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:45.824389 10906 raft_consensus.cc:2272] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:45.868995 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.869295 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:45.869295 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:45.893904 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:46.086855 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.087064 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.087335 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:46.111114 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:46.377331 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.377666 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.378018 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:46.400883 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:46.781431 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.782339 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:46.782487 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:46.803894 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:47.128842 18139 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 24da0820c3c14d0f8507938315d91e91)
I20260811 01:57:47.129009 18139 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:47.129315 18140 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 24da0820c3c14d0f8507938315d91e91)
I20260811 01:57:47.129431 18140 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:47.129664 18139 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:47.129958 18140 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:47.134302 17791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "d6cd6377ae804351a400f085ec7bfb33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91" is_pre_election: true
I20260811 01:57:47.134367 17539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "d6cd6377ae804351a400f085ec7bfb33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce8f83ac34584178a56c1937d383b73e" is_pre_election: true
I20260811 01:57:47.134506 17539 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6cd6377ae804351a400f085ec7bfb33 in term 1.
I20260811 01:57:47.134548 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33" is_pre_election: true
I20260811 01:57:47.134640 17666 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce8f83ac34584178a56c1937d383b73e in term 1.
I20260811 01:57:47.134377 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91" is_pre_election: true
I20260811 01:57:47.134812 17609 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce8f83ac34584178a56c1937d383b73e, d6cd6377ae804351a400f085ec7bfb33; no voters:
I20260811 01:57:47.134984 18140 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:47.135046 18140 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 24da0820c3c14d0f8507938315d91e91)
W20260811 01:57:47.135071 17606 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:47.135089 18140 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 1 FOLLOWER]: Advancing to term 2
W20260811 01:57:47.135062 17484 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:47.135421 17483 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce8f83ac34584178a56c1937d383b73e, d6cd6377ae804351a400f085ec7bfb33; no voters: 24da0820c3c14d0f8507938315d91e91
I20260811 01:57:47.135586 18139 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:47.135643 18139 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 24da0820c3c14d0f8507938315d91e91)
I20260811 01:57:47.135687 18139 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:47.136260 18140 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:47.136533 18139 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:47.136610 18140 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 election: Requested vote from peers ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:47.136888 18139 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 election: Requested vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:47.136922 17539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "d6cd6377ae804351a400f085ec7bfb33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce8f83ac34584178a56c1937d383b73e"
I20260811 01:57:47.137027 17539 raft_consensus.cc:2393] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d6cd6377ae804351a400f085ec7bfb33 in current term 2: Already voted for candidate ce8f83ac34584178a56c1937d383b73e in this term.
I20260811 01:57:47.137063 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "d6cd6377ae804351a400f085ec7bfb33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91"
I20260811 01:57:47.137161 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33"
I20260811 01:57:47.137274 17666 raft_consensus.cc:2393] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ce8f83ac34584178a56c1937d383b73e in current term 2: Already voted for candidate d6cd6377ae804351a400f085ec7bfb33 in this term.
W20260811 01:57:47.137341 17606 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:47.137595 17609 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6cd6377ae804351a400f085ec7bfb33; no voters: 24da0820c3c14d0f8507938315d91e91, ce8f83ac34584178a56c1937d383b73e
I20260811 01:57:47.137745 18140 raft_consensus.cc:2749] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 01:57:47.137827 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91"
W20260811 01:57:47.138087 17484 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:47.138150 17484 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce8f83ac34584178a56c1937d383b73e; no voters: 24da0820c3c14d0f8507938315d91e91, d6cd6377ae804351a400f085ec7bfb33
I20260811 01:57:47.138267 18139 raft_consensus.cc:2749] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 01:57:47.351652 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:47.352345 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:47.352711 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 2 | -1 | Yes
B | A B C | 2 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:47.375022 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:48.178445 17522 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:48.178480 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:48.180289 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 0 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 2 | -1 | Yes
B | A B C | 2 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:48.201459 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:48.741250 18232 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:48.741425 18232 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:48.741909 18232 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:48.742419 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91" is_pre_election: true
I20260811 01:57:48.742419 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33" is_pre_election: true
I20260811 01:57:48.742615 17666 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce8f83ac34584178a56c1937d383b73e in term 2.
I20260811 01:57:48.742905 17483 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce8f83ac34584178a56c1937d383b73e, d6cd6377ae804351a400f085ec7bfb33; no voters:
W20260811 01:57:48.743039 17484 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:48.743112 18232 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:57:48.743189 18232 raft_consensus.cc:493] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:48.743235 18232 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:57:48.744274 18232 raft_consensus.cc:515] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:48.744616 18232 leader_election.cc:290] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 election: Requested vote from peers d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351), 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415)
I20260811 01:57:48.744933 17666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6cd6377ae804351a400f085ec7bfb33"
I20260811 01:57:48.744973 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3aabcc3e274eebb1c1e30edd9a0845" candidate_uuid: "ce8f83ac34584178a56c1937d383b73e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24da0820c3c14d0f8507938315d91e91"
I20260811 01:57:48.745031 17666 raft_consensus.cc:3060] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 2 FOLLOWER]: Advancing to term 3
W20260811 01:57:48.745253 17484 leader_election.cc:343] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Illegal state: cannot vote while shut down
I20260811 01:57:48.745826 17666 raft_consensus.cc:2468] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce8f83ac34584178a56c1937d383b73e in term 3.
I20260811 01:57:48.746083 17483 leader_election.cc:304] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce8f83ac34584178a56c1937d383b73e, d6cd6377ae804351a400f085ec7bfb33; no voters: 24da0820c3c14d0f8507938315d91e91
I20260811 01:57:48.746238 18232 raft_consensus.cc:2804] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:57:48.746403 18232 raft_consensus.cc:697] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 LEADER]: Becoming Leader. State: Replica: ce8f83ac34584178a56c1937d383b73e, State: Running, Role: LEADER
I20260811 01:57:48.746562 18232 consensus_queue.cc:237] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:48.748042 17362 catalog_manager.cc:5719] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e reported cstate change: term changed from 1 to 3, leader changed from 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) to ce8f83ac34584178a56c1937d383b73e (127.10.166.129). New cstate: current_term: 3 leader_uuid: "ce8f83ac34584178a56c1937d383b73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } } }
I20260811 01:57:49.128638 17666 raft_consensus.cc:1275] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Refusing update from remote peer ce8f83ac34584178a56c1937d383b73e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260811 01:57:49.129153 18232 consensus_queue.cc:1048] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260811 01:57:49.132220 17484 consensus_peers.cc:597] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e -> Peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Couldn't send request to peer 24da0820c3c14d0f8507938315d91e91. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:49.132426 18233 raft_consensus.cc:1012] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Attempting to remove follower 24da0820c3c14d0f8507938315d91e91 from the Raft config. Reason: The tablet replica hosted on peer 24da0820c3c14d0f8507938315d91e91 has failed
I20260811 01:57:49.132869 18233 consensus_queue.cc:237] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } }
I20260811 01:57:49.133816 17666 raft_consensus.cc:1275] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Refusing update from remote peer ce8f83ac34584178a56c1937d383b73e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260811 01:57:49.134146 18232 consensus_queue.cc:1048] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:49.135422 18233 raft_consensus.cc:2955] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } }
I20260811 01:57:49.135867 17666 raft_consensus.cc:2955] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } }
I20260811 01:57:49.137732 17362 catalog_manager.cc:5719] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e reported cstate change: config changed from index -1 to 3, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) evicted. New cstate: current_term: 3 leader_uuid: "ce8f83ac34584178a56c1937d383b73e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } }
I20260811 01:57:49.139918 17773 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 with delete_type TABLET_DATA_TOMBSTONED (TS 24da0820c3c14d0f8507938315d91e91 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:55722
I20260811 01:57:49.140260 18243 ts_tablet_manager.cc:1916] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 01:57:49.140413 17539 consensus_queue.cc:237] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:49.141445 17666 raft_consensus.cc:1275] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Refusing update from remote peer ce8f83ac34584178a56c1937d383b73e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260811 01:57:49.141585 17484 consensus_peers.cc:597] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e -> Peer 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): Couldn't send request to peer 24da0820c3c14d0f8507938315d91e91. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:49.141786 18232 consensus_queue.cc:1048] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:49.142362 18243 ts_tablet_manager.cc:1929] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260811 01:57:49.142429 18243 log.cc:1199] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1786413426438645-10906-0/minicluster-data/ts-2-root/wals/1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:49.142917 17343 catalog_manager.cc:5050] TS 24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): tablet 1d3aabcc3e274eebb1c1e30edd9a0845 (table test-workload [id=e65ac1d8cb0645f389cdf77339884cac]) successfully deleted
I20260811 01:57:49.143047 18233 raft_consensus.cc:2955] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } } }
I20260811 01:57:49.143468 17666 raft_consensus.cc:2955] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } } }
I20260811 01:57:49.144244 17344 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:VOTER RPC for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 with cas_config_opid_index 3: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260811 01:57:49.145043 17362 catalog_manager.cc:5719] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e reported cstate change: config changed from index 3 to 4, VOTER 24da0820c3c14d0f8507938315d91e91 (127.10.166.131) added. New cstate: current_term: 3 leader_uuid: "ce8f83ac34584178a56c1937d383b73e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } } }
I20260811 01:57:49.244400 17522 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:49.244539 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:49.244544 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 1 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 1d3aabcc3e274eebb1c1e30edd9a0845 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
d6cd6377ae804351a400f085ec7bfb33 (127.10.166.130:33351): RUNNING
ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507): RUNNING [LEADER]
24da0820c3c14d0f8507938315d91e91 (127.10.166.131:34415): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = d6cd6377ae804351a400f085ec7bfb33
B = ce8f83ac34584178a56c1937d383b73e
C = 24da0820c3c14d0f8507938315d91e91
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 4 | Yes
B | A B* C | 3 | 4 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:49.267644 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:49.621090 18291 ts_tablet_manager.cc:933] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Initiating tablet copy from peer ce8f83ac34584178a56c1937d383b73e (127.10.166.129:38507)
I20260811 01:57:49.621485 18291 tablet_copy_client.cc:323] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet copy: Beginning tablet copy session from remote peer at address 127.10.166.129:38507
I20260811 01:57:49.622031 17554 tablet_copy_service.cc:140] P ce8f83ac34584178a56c1937d383b73e: Received BeginTabletCopySession request for tablet 1d3aabcc3e274eebb1c1e30edd9a0845 from peer 24da0820c3c14d0f8507938315d91e91 ({username='slave'} at 127.0.0.1:52600)
I20260811 01:57:49.622141 17554 tablet_copy_service.cc:161] P ce8f83ac34584178a56c1937d383b73e: Beginning new tablet copy session on tablet 1d3aabcc3e274eebb1c1e30edd9a0845 from peer 24da0820c3c14d0f8507938315d91e91 at {username='slave'} at 127.0.0.1:52600: session id = 24da0820c3c14d0f8507938315d91e91-1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:49.622960 17554 tablet_copy_source_session.cc:215] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:57:49.623636 18291 ts_tablet_manager.cc:1916] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260811 01:57:49.625422 18291 ts_tablet_manager.cc:1929] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260811 01:57:49.625602 18291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3aabcc3e274eebb1c1e30edd9a0845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:49.626514 18291 tablet_copy_client.cc:806] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet copy: Starting download of 0 data blocks...
I20260811 01:57:49.626682 18291 tablet_copy_client.cc:670] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet copy: Starting download of 1 WAL segments...
I20260811 01:57:49.627669 18291 tablet_copy_client.cc:538] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:57:49.628644 18291 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Bootstrap starting.
I20260811 01:57:49.632025 18291 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:49.632237 18291 tablet_bootstrap.cc:492] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Bootstrap complete.
I20260811 01:57:49.632385 18291 ts_tablet_manager.cc:1403] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 01:57:49.632790 18291 raft_consensus.cc:359] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:49.632882 18291 raft_consensus.cc:740] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24da0820c3c14d0f8507938315d91e91, State: Initialized, Role: FOLLOWER
I20260811 01:57:49.633006 18291 consensus_queue.cc:260] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d6cd6377ae804351a400f085ec7bfb33" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 33351 } } peers { permanent_uuid: "ce8f83ac34584178a56c1937d383b73e" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 38507 } } peers { permanent_uuid: "24da0820c3c14d0f8507938315d91e91" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 34415 } }
I20260811 01:57:49.633304 18291 ts_tablet_manager.cc:1434] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:49.633744 17554 tablet_copy_service.cc:342] P ce8f83ac34584178a56c1937d383b73e: Request end of tablet copy session 24da0820c3c14d0f8507938315d91e91-1d3aabcc3e274eebb1c1e30edd9a0845 received from {username='slave'} at 127.0.0.1:52600
I20260811 01:57:49.633817 17554 tablet_copy_service.cc:434] P ce8f83ac34584178a56c1937d383b73e: ending tablet copy session 24da0820c3c14d0f8507938315d91e91-1d3aabcc3e274eebb1c1e30edd9a0845 on tablet 1d3aabcc3e274eebb1c1e30edd9a0845 with peer 24da0820c3c14d0f8507938315d91e91
I20260811 01:57:50.041008 17790 raft_consensus.cc:1217] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20260811 01:57:50.309737 17652 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:50.309901 17522 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:50.310475 17773 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
0e9aba7ccdc44134a3fd532a5f8c7663 | 127.10.166.190:32845 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
24da0820c3c14d0f8507938315d91e91 | 127.10.166.131:34415 | HEALTHY | <none> | 0 | 0
ce8f83ac34584178a56c1937d383b73e | 127.10.166.129:38507 | HEALTHY | <none> | 1 | 0
d6cd6377ae804351a400f085ec7bfb33 | 127.10.166.130:33351 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 01:57:50.332221 10906 log_verifier.cc:126] Checking tablet 1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:50.349853 10906 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 1d3aabcc3e274eebb1c1e30edd9a0845
I20260811 01:57:50.350518 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:50.353893 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:50.354063 10906 tablet_replica.cc:333] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e: stopping tablet replica
I20260811 01:57:50.354156 10906 raft_consensus.cc:2243] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 LEADER]: Raft consensus shutting down.
I20260811 01:57:50.354318 10906 raft_consensus.cc:2272] T 1d3aabcc3e274eebb1c1e30edd9a0845 P ce8f83ac34584178a56c1937d383b73e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:50.367659 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:50.369529 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:50.373580 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:50.373757 10906 tablet_replica.cc:333] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33: stopping tablet replica
I20260811 01:57:50.373855 10906 raft_consensus.cc:2243] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:50.373961 10906 raft_consensus.cc:2272] T 1d3aabcc3e274eebb1c1e30edd9a0845 P d6cd6377ae804351a400f085ec7bfb33 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:50.387104 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:50.388865 10906 tablet_server.cc:179] TabletServer@127.10.166.131:0 shutting down...
I20260811 01:57:50.392167 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:50.392334 10906 tablet_replica.cc:333] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91: stopping tablet replica
I20260811 01:57:50.392427 10906 raft_consensus.cc:2243] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:50.392536 10906 raft_consensus.cc:2272] T 1d3aabcc3e274eebb1c1e30edd9a0845 P 24da0820c3c14d0f8507938315d91e91 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:50.405308 10906 tablet_server.cc:196] TabletServer@127.10.166.131:0 shutdown complete.
I20260811 01:57:50.406922 10906 master.cc:562] Master@127.10.166.190:32845 shutting down...
I20260811 01:57:50.409969 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:50.410087 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:50.410132 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0e9aba7ccdc44134a3fd532a5f8c7663: stopping tablet replica
I20260811 01:57:50.421993 10906 master.cc:584] Master@127.10.166.190:32845 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (5394 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260811 01:57:50.427760 10906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.166.190:36097
I20260811 01:57:50.428084 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:50.429531 18344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.429555 18343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.429642 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:50.429664 18346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.429901 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:50.429946 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:50.429966 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413470429966 us; error 0 us; skew 500 ppm
I20260811 01:57:50.430687 10906 webserver.cc:533] Webserver started at http://127.10.166.190:37627/ using document root <none> and password file <none>
I20260811 01:57:50.430823 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:50.430871 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:50.430940 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:50.431295 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/master-0-root/instance:
uuid: "2771ba0f778f4ff8ab7b279a7be78839"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.432742 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:50.433607 18355 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.433825 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:50.433889 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/master-0-root
uuid: "2771ba0f778f4ff8ab7b279a7be78839"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.433957 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:50.446336 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:50.446745 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:50.450960 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.190:36097
I20260811 01:57:50.453679 18435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.190:36097 every 8 connection(s)
I20260811 01:57:50.454118 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:50.455890 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839: Bootstrap starting.
I20260811 01:57:50.456645 18436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:50.457628 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839: No bootstrap required, opened a new log
I20260811 01:57:50.457988 18436 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER }
I20260811 01:57:50.458069 18436 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:50.458096 18436 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771ba0f778f4ff8ab7b279a7be78839, State: Initialized, Role: FOLLOWER
I20260811 01:57:50.458206 18436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER }
I20260811 01:57:50.458266 18436 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:57:50.458300 18436 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:57:50.458333 18436 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:50.458952 18436 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER }
I20260811 01:57:50.459065 18436 leader_election.cc:304] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2771ba0f778f4ff8ab7b279a7be78839; no voters:
I20260811 01:57:50.459213 18436 leader_election.cc:290] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:57:50.459331 18441 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:50.459563 18441 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 1 LEADER]: Becoming Leader. State: Replica: 2771ba0f778f4ff8ab7b279a7be78839, State: Running, Role: LEADER
I20260811 01:57:50.459631 18436 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:57:50.459728 18441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER }
I20260811 01:57:50.460181 18445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2771ba0f778f4ff8ab7b279a7be78839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER } }
I20260811 01:57:50.460211 18446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2771ba0f778f4ff8ab7b279a7be78839. Latest consensus state: current_term: 1 leader_uuid: "2771ba0f778f4ff8ab7b279a7be78839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2771ba0f778f4ff8ab7b279a7be78839" member_type: VOTER } }
I20260811 01:57:50.460359 18446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:50.460623 18456 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260811 01:57:50.460847 18445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [sys.catalog]: This master's current role is: LEADER
I20260811 01:57:50.461496 18456 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260811 01:57:50.461654 10906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260811 01:57:50.463268 18456 catalog_manager.cc:1383] Generated new cluster ID: 478087f45c044662b827296ae885491e
I20260811 01:57:50.463327 18456 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260811 01:57:50.484302 18456 catalog_manager.cc:1406] Generated new certificate authority record
I20260811 01:57:50.484946 18456 catalog_manager.cc:1540] Loading token signing keys...
I20260811 01:57:50.493193 18456 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839: Generated new TSK 0
I20260811 01:57:50.493350 18456 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260811 01:57:50.526209 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:50.528155 18476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.528247 10906 server_base.cc:1061] running on GCE node
W20260811 01:57:50.528177 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.528152 18473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.528563 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:50.528604 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:50.528620 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413470528619 us; error 0 us; skew 500 ppm
I20260811 01:57:50.529402 10906 webserver.cc:533] Webserver started at http://127.10.166.129:38227/ using document root <none> and password file <none>
I20260811 01:57:50.529556 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:50.529601 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:50.529671 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:50.530053 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-0-root/instance:
uuid: "d5ab76853ad3402bbfd599a73683f7fe"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.531454 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 01:57:50.532351 18486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.532562 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.532627 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-0-root
uuid: "d5ab76853ad3402bbfd599a73683f7fe"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.532696 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:50.554615 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:50.554937 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:50.555209 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:50.555670 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:50.555733 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.555787 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:50.555815 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.559636 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.129:32835
I20260811 01:57:50.559693 18587 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.129:32835 every 8 connection(s)
I20260811 01:57:50.560410 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:50.562017 18599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.562067 18600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.562331 18603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.562400 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:50.562605 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:50.562647 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:50.562668 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413470562668 us; error 0 us; skew 500 ppm
I20260811 01:57:50.563508 10906 webserver.cc:533] Webserver started at http://127.10.166.130:44029/ using document root <none> and password file <none>
I20260811 01:57:50.563659 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:50.563764 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:50.563849 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:50.564278 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-1-root/instance:
uuid: "3491269ab9ef47d4804468f6f50a43fc"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.565066 18593 heartbeater.cc:344] Connected to a master server at 127.10.166.190:36097
I20260811 01:57:50.565158 18593 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:50.565366 18593 heartbeater.cc:507] Master 127.10.166.190:36097 requested a full tablet report, sending...
I20260811 01:57:50.565953 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:50.566066 18380 ts_manager.cc:194] Registered new tserver with Master: d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129:32835)
I20260811 01:57:50.566998 18614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.567006 18380 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46296
I20260811 01:57:50.567314 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.567422 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-1-root
uuid: "3491269ab9ef47d4804468f6f50a43fc"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.567497 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:50.575443 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:50.575785 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:50.576043 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:50.576458 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:50.576495 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.576529 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:50.576556 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.580796 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.130:35045
I20260811 01:57:50.580835 18714 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.130:35045 every 8 connection(s)
I20260811 01:57:50.581547 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:50.583292 18722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.583313 18725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.583482 18721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.583609 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:50.583783 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:50.583822 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:50.583845 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413470583845 us; error 0 us; skew 500 ppm
I20260811 01:57:50.584630 10906 webserver.cc:533] Webserver started at http://127.10.166.131:33793/ using document root <none> and password file <none>
I20260811 01:57:50.584774 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:50.584831 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:50.584952 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:50.585331 18715 heartbeater.cc:344] Connected to a master server at 127.10.166.190:36097
I20260811 01:57:50.585431 18715 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:50.585479 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root/instance:
uuid: "b79b8a503e864a07ae4c670f5e41d9f2"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.585630 18715 heartbeater.cc:507] Master 127.10.166.190:36097 requested a full tablet report, sending...
I20260811 01:57:50.586203 18380 ts_manager.cc:194] Registered new tserver with Master: 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045)
I20260811 01:57:50.586690 18380 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46302
I20260811 01:57:50.587010 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 01:57:50.587872 18733 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.588074 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.588140 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root
uuid: "b79b8a503e864a07ae4c670f5e41d9f2"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.588205 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:50.604432 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:50.604830 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:50.605114 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:50.605564 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:50.605602 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.605645 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:50.605674 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.610098 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.131:40679
I20260811 01:57:50.610157 18838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.131:40679 every 8 connection(s)
I20260811 01:57:50.610855 10906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260811 01:57:50.612546 18843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.612634 18842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:57:50.612874 18847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:57:50.612973 10906 server_base.cc:1061] running on GCE node
I20260811 01:57:50.613221 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260811 01:57:50.613282 10906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260811 01:57:50.613304 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786413470613303 us; error 0 us; skew 500 ppm
I20260811 01:57:50.614324 10906 webserver.cc:533] Webserver started at http://127.10.166.132:33625/ using document root <none> and password file <none>
I20260811 01:57:50.614497 10906 fs_manager.cc:362] Metadata directory not provided
I20260811 01:57:50.614557 10906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:57:50.614636 10906 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:57:50.614864 18839 heartbeater.cc:344] Connected to a master server at 127.10.166.190:36097
I20260811 01:57:50.614943 18839 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:50.615069 10906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-3-root/instance:
uuid: "142af6a7a51b465882ddef279ba1b17e"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.615128 18839 heartbeater.cc:507] Master 127.10.166.190:36097 requested a full tablet report, sending...
I20260811 01:57:50.615744 18380 ts_manager.cc:194] Registered new tserver with Master: b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679)
I20260811 01:57:50.616236 18380 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46306
I20260811 01:57:50.616655 10906 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:50.617542 18855 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.617812 10906 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.617898 10906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-3-root
uuid: "142af6a7a51b465882ddef279ba1b17e"
format_stamp: "Formatted at 2026-08-11 01:57:50 on dist-test-slave-1l3l"
I20260811 01:57:50.617966 10906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:57:50.627144 10906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260811 01:57:50.627475 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:57:50.627785 10906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260811 01:57:50.628211 10906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:57:50.628248 10906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.628280 10906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260811 01:57:50.628309 10906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:57:50.632674 10906 rpc_server.cc:307] RPC server started. Bound to: 127.10.166.132:42437
I20260811 01:57:50.632738 18957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.166.132:42437 every 8 connection(s)
I20260811 01:57:50.636767 18958 heartbeater.cc:344] Connected to a master server at 127.10.166.190:36097
I20260811 01:57:50.636852 18958 heartbeater.cc:461] Registering TS with master...
I20260811 01:57:50.637040 18958 heartbeater.cc:507] Master 127.10.166.190:36097 requested a full tablet report, sending...
I20260811 01:57:50.637610 18380 ts_manager.cc:194] Registered new tserver with Master: 142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437)
I20260811 01:57:50.637678 10906 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.004574706s
I20260811 01:57:50.638249 18380 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46316
I20260811 01:57:50.649499 18380 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46338:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:57:50.658846 18900 tablet_service.cc:1511] Processing CreateTablet for tablet 9c19dc00e54743029dfe71fab44942f1 (DEFAULT_TABLE table=test-workload [id=25c4091b96574cdb81e2a8c810cf4a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:50.659176 18900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c19dc00e54743029dfe71fab44942f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:50.659296 18657 tablet_service.cc:1511] Processing CreateTablet for tablet 9c19dc00e54743029dfe71fab44942f1 (DEFAULT_TABLE table=test-workload [id=25c4091b96574cdb81e2a8c810cf4a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:50.659535 18657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c19dc00e54743029dfe71fab44942f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:50.661581 18987 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: Bootstrap starting.
I20260811 01:57:50.662112 18783 tablet_service.cc:1511] Processing CreateTablet for tablet 9c19dc00e54743029dfe71fab44942f1 (DEFAULT_TABLE table=test-workload [id=25c4091b96574cdb81e2a8c810cf4a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:57:50.662367 18783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c19dc00e54743029dfe71fab44942f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:50.662968 18987 tablet_bootstrap.cc:654] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:50.663779 18988 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: Bootstrap starting.
I20260811 01:57:50.664150 18987 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: No bootstrap required, opened a new log
I20260811 01:57:50.664268 18987 ts_tablet_manager.cc:1403] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:57:50.664736 18987 raft_consensus.cc:359] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.664829 18987 raft_consensus.cc:385] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:50.664834 18988 tablet_bootstrap.cc:654] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:50.664851 18987 raft_consensus.cc:740] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142af6a7a51b465882ddef279ba1b17e, State: Initialized, Role: FOLLOWER
I20260811 01:57:50.665014 18987 consensus_queue.cc:260] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.665313 18987 ts_tablet_manager.cc:1434] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.665386 18958 heartbeater.cc:499] Master 127.10.166.190:36097 was elected leader, sending a full tablet report...
I20260811 01:57:50.665604 18991 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Bootstrap starting.
I20260811 01:57:50.665915 18988 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: No bootstrap required, opened a new log
I20260811 01:57:50.666005 18988 ts_tablet_manager.cc:1403] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:50.666450 18988 raft_consensus.cc:359] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.666558 18988 raft_consensus.cc:385] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:50.666599 18988 raft_consensus.cc:740] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3491269ab9ef47d4804468f6f50a43fc, State: Initialized, Role: FOLLOWER
I20260811 01:57:50.666783 18991 tablet_bootstrap.cc:654] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Neither blocks nor log segments found. Creating new log.
I20260811 01:57:50.666731 18988 consensus_queue.cc:260] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.667152 18988 ts_tablet_manager.cc:1434] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 01:57:50.667303 18715 heartbeater.cc:499] Master 127.10.166.190:36097 was elected leader, sending a full tablet report...
I20260811 01:57:50.667948 18991 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: No bootstrap required, opened a new log
I20260811 01:57:50.668020 18991 ts_tablet_manager.cc:1403] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:57:50.668380 18991 raft_consensus.cc:359] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.668480 18991 raft_consensus.cc:385] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:57:50.668519 18991 raft_consensus.cc:740] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79b8a503e864a07ae4c670f5e41d9f2, State: Initialized, Role: FOLLOWER
I20260811 01:57:50.668638 18991 consensus_queue.cc:260] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.668941 18991 ts_tablet_manager.cc:1434] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 01:57:50.668973 18839 heartbeater.cc:499] Master 127.10.166.190:36097 was elected leader, sending a full tablet report...
I20260811 01:57:50.698959 19000 raft_consensus.cc:493] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:57:50.699060 19000 raft_consensus.cc:515] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.699529 19000 leader_election.cc:290] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437), 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045)
I20260811 01:57:50.702900 18916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142af6a7a51b465882ddef279ba1b17e" is_pre_election: true
I20260811 01:57:50.702960 18672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3491269ab9ef47d4804468f6f50a43fc" is_pre_election: true
I20260811 01:57:50.703023 18916 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79b8a503e864a07ae4c670f5e41d9f2 in term 0.
I20260811 01:57:50.703068 18672 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79b8a503e864a07ae4c670f5e41d9f2 in term 0.
I20260811 01:57:50.703329 18736 leader_election.cc:304] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142af6a7a51b465882ddef279ba1b17e, b79b8a503e864a07ae4c670f5e41d9f2; no voters:
I20260811 01:57:50.703507 19000 raft_consensus.cc:2804] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:57:50.703567 19000 raft_consensus.cc:493] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:57:50.703613 19000 raft_consensus.cc:3060] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:50.704427 19000 raft_consensus.cc:515] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.704763 19000 leader_election.cc:290] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [CANDIDATE]: Term 1 election: Requested vote from peers 142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437), 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045)
I20260811 01:57:50.705044 18916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142af6a7a51b465882ddef279ba1b17e"
I20260811 01:57:50.705055 18672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3491269ab9ef47d4804468f6f50a43fc"
I20260811 01:57:50.705143 18916 raft_consensus.cc:3060] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:50.705157 18672 raft_consensus.cc:3060] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:57:50.705967 18916 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79b8a503e864a07ae4c670f5e41d9f2 in term 1.
I20260811 01:57:50.705972 18672 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79b8a503e864a07ae4c670f5e41d9f2 in term 1.
I20260811 01:57:50.706220 18736 leader_election.cc:304] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142af6a7a51b465882ddef279ba1b17e, b79b8a503e864a07ae4c670f5e41d9f2; no voters:
I20260811 01:57:50.706426 19000 raft_consensus.cc:2804] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:57:50.706620 19000 raft_consensus.cc:697] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 1 LEADER]: Becoming Leader. State: Replica: b79b8a503e864a07ae4c670f5e41d9f2, State: Running, Role: LEADER
I20260811 01:57:50.706750 19000 consensus_queue.cc:237] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:50.708137 18380 catalog_manager.cc:5719] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131). New cstate: current_term: 1 leader_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:50.757985 18783 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:50.758072 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:50.758773 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:50.760601 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 1 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): RUNNING [LEADER]
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:50.785933 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:50.982318 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:50.983337 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:50.983511 18783 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:50.983527 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 1 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): RUNNING [LEADER]
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:51.010309 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:51.102934 19000 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:51.106364 19002 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:57:51.277534 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.278031 18783 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:51.278096 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.279170 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 1 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 01:57:51.303964 10906 log_verifier.cc:126] Checking tablet 9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:51.326081 10906 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:51.326254 10906 tablet_replica.cc:333] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: stopping tablet replica
I20260811 01:57:51.326362 10906 raft_consensus.cc:2243] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:51.326606 10906 raft_consensus.cc:2272] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:51.369469 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.369884 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.370559 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.371070 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:51.398733 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:51.567974 18593 heartbeater.cc:499] Master 127.10.166.190:36097 was elected leader, sending a full tablet report...
I20260811 01:57:51.590605 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.591146 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.591203 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.592056 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:51.617498 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:51.884016 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.884912 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.885545 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:51.886170 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:51.911092 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:52.290640 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:52.290640 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:52.292451 18657 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:52.300818 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 0 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:52.323161 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:52.606503 19298 raft_consensus.cc:493] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b79b8a503e864a07ae4c670f5e41d9f2)
I20260811 01:57:52.606632 19298 raft_consensus.cc:515] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:52.607168 19298 leader_election.cc:290] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679), 142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437)
I20260811 01:57:52.607494 19301 raft_consensus.cc:493] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b79b8a503e864a07ae4c670f5e41d9f2)
I20260811 01:57:52.607815 19301 raft_consensus.cc:515] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:52.609472 19301 leader_election.cc:290] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679), 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045)
I20260811 01:57:52.611660 18797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "3491269ab9ef47d4804468f6f50a43fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" is_pre_election: true
I20260811 01:57:52.611974 18916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "3491269ab9ef47d4804468f6f50a43fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142af6a7a51b465882ddef279ba1b17e" is_pre_election: true
I20260811 01:57:52.612098 18916 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3491269ab9ef47d4804468f6f50a43fc in term 1.
W20260811 01:57:52.612439 18617 leader_election.cc:343] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): Illegal state: cannot vote while shut down
I20260811 01:57:52.612576 18617 leader_election.cc:304] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142af6a7a51b465882ddef279ba1b17e, 3491269ab9ef47d4804468f6f50a43fc; no voters: b79b8a503e864a07ae4c670f5e41d9f2
I20260811 01:57:52.612733 19298 raft_consensus.cc:2804] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:57:52.612785 19298 raft_consensus.cc:493] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Starting leader election (detected failure of leader b79b8a503e864a07ae4c670f5e41d9f2)
I20260811 01:57:52.612818 18672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "142af6a7a51b465882ddef279ba1b17e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3491269ab9ef47d4804468f6f50a43fc" is_pre_election: true
I20260811 01:57:52.612833 19298 raft_consensus.cc:3060] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:52.613023 18797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "142af6a7a51b465882ddef279ba1b17e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" is_pre_election: true
W20260811 01:57:52.613384 18857 leader_election.cc:343] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): Illegal state: cannot vote while shut down
I20260811 01:57:52.613950 19298 raft_consensus.cc:515] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:52.614154 18672 raft_consensus.cc:2393] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 142af6a7a51b465882ddef279ba1b17e in current term 2: Already voted for candidate 3491269ab9ef47d4804468f6f50a43fc in this term.
I20260811 01:57:52.614300 19298 leader_election.cc:290] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 election: Requested vote from peers b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679), 142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437)
I20260811 01:57:52.614630 18916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "3491269ab9ef47d4804468f6f50a43fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "142af6a7a51b465882ddef279ba1b17e"
I20260811 01:57:52.614636 18797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c19dc00e54743029dfe71fab44942f1" candidate_uuid: "3491269ab9ef47d4804468f6f50a43fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b79b8a503e864a07ae4c670f5e41d9f2"
I20260811 01:57:52.614739 18916 raft_consensus.cc:3060] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:57:52.614766 18859 leader_election.cc:304] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 142af6a7a51b465882ddef279ba1b17e; no voters: 3491269ab9ef47d4804468f6f50a43fc, b79b8a503e864a07ae4c670f5e41d9f2
W20260811 01:57:52.614989 18617 leader_election.cc:343] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): Illegal state: cannot vote while shut down
I20260811 01:57:52.615542 18916 raft_consensus.cc:2468] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3491269ab9ef47d4804468f6f50a43fc in term 2.
I20260811 01:57:52.615640 19301 raft_consensus.cc:2749] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:57:52.615831 18617 leader_election.cc:304] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 142af6a7a51b465882ddef279ba1b17e, 3491269ab9ef47d4804468f6f50a43fc; no voters: b79b8a503e864a07ae4c670f5e41d9f2
I20260811 01:57:52.616004 19298 raft_consensus.cc:2804] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:57:52.616181 19298 raft_consensus.cc:697] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 LEADER]: Becoming Leader. State: Replica: 3491269ab9ef47d4804468f6f50a43fc, State: Running, Role: LEADER
I20260811 01:57:52.616336 19298 consensus_queue.cc:237] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:52.617899 18379 catalog_manager.cc:5719] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc reported cstate change: term changed from 1 to 2, leader changed from b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131) to 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130). New cstate: current_term: 2 leader_uuid: "3491269ab9ef47d4804468f6f50a43fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79b8a503e864a07ae4c670f5e41d9f2" member_type: VOTER last_known_addr { host: "127.10.166.131" port: 40679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:52.879040 18657 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:52.879081 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:52.879083 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:52.879698 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 1 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING [LEADER]
All reported replicas are:
A = b79b8a503e864a07ae4c670f5e41d9f2
B = 142af6a7a51b465882ddef279ba1b17e
C = 3491269ab9ef47d4804468f6f50a43fc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | [config not available] | | |
B | A B C | 2 | -1 | Yes
C | A B C* | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:52.908078 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:53.034798 18617 consensus_queue.cc:579] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: The tablet replica hosted on peer b79b8a503e864a07ae4c670f5e41d9f2 has failed
W20260811 01:57:53.034937 18617 consensus_peers.cc:597] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc -> Peer b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): Couldn't send request to peer b79b8a503e864a07ae4c670f5e41d9f2. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:53.055395 18916 raft_consensus.cc:1275] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Refusing update from remote peer 3491269ab9ef47d4804468f6f50a43fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260811 01:57:53.055848 19298 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260811 01:57:53.059891 18672 consensus_queue.cc:237] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } }
I20260811 01:57:53.060772 18916 raft_consensus.cc:1275] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Refusing update from remote peer 3491269ab9ef47d4804468f6f50a43fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 01:57:53.061108 19298 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 01:57:53.062438 19312 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } }
I20260811 01:57:53.062839 18916 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } }
I20260811 01:57:53.063938 18358 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 9c19dc00e54743029dfe71fab44942f1 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260811 01:57:53.064488 18379 catalog_manager.cc:5719] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc reported cstate change: config changed from index -1 to 3, VOTER b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131) evicted. New cstate: current_term: 2 leader_uuid: "3491269ab9ef47d4804468f6f50a43fc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } health_report { overall_health: HEALTHY } } }
I20260811 01:57:53.066569 18783 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c19dc00e54743029dfe71fab44942f1 with delete_type TABLET_DATA_TOMBSTONED (TS b79b8a503e864a07ae4c670f5e41d9f2 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42748
I20260811 01:57:53.066907 19371 ts_tablet_manager.cc:1916] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260811 01:57:53.067118 18672 consensus_queue.cc:237] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } }
I20260811 01:57:53.068326 18916 raft_consensus.cc:1275] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Refusing update from remote peer 3491269ab9ef47d4804468f6f50a43fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260811 01:57:53.068964 19371 ts_tablet_manager.cc:1929] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260811 01:57:53.069070 19371 log.cc:1199] T 9c19dc00e54743029dfe71fab44942f1 P b79b8a503e864a07ae4c670f5e41d9f2: Deleting WAL directory at /tmp/dist-test-taskarUZWu/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1786413426438645-10906-0/minicluster-data/ts-2-root/wals/9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:53.069037 19312 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260811 01:57:53.069705 18357 catalog_manager.cc:5050] TS b79b8a503e864a07ae4c670f5e41d9f2 (127.10.166.131:40679): tablet 9c19dc00e54743029dfe71fab44942f1 (table test-workload [id=25c4091b96574cdb81e2a8c810cf4a13]) successfully deleted
I20260811 01:57:53.070478 19298 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } } }
I20260811 01:57:53.070865 18916 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } } }
W20260811 01:57:53.072024 18619 consensus_peers.cc:597] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc -> Peer d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129:32835): Couldn't send request to peer d5ab76853ad3402bbfd599a73683f7fe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c19dc00e54743029dfe71fab44942f1. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:57:53.072088 18358 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9c19dc00e54743029dfe71fab44942f1 with cas_config_opid_index 3: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260811 01:57:53.072595 18379 catalog_manager.cc:5719] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc reported cstate change: config changed from index 3 to 4, NON_VOTER d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) added. New cstate: current_term: 2 leader_uuid: "3491269ab9ef47d4804468f6f50a43fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260811 01:57:53.603302 19375 ts_tablet_manager.cc:933] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Initiating tablet copy from peer 3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045)
I20260811 01:57:53.603610 19375 tablet_copy_client.cc:323] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: tablet copy: Beginning tablet copy session from remote peer at address 127.10.166.130:35045
I20260811 01:57:53.606987 18687 tablet_copy_service.cc:140] P 3491269ab9ef47d4804468f6f50a43fc: Received BeginTabletCopySession request for tablet 9c19dc00e54743029dfe71fab44942f1 from peer d5ab76853ad3402bbfd599a73683f7fe ({username='slave'} at 127.0.0.1:59982)
I20260811 01:57:53.607092 18687 tablet_copy_service.cc:161] P 3491269ab9ef47d4804468f6f50a43fc: Beginning new tablet copy session on tablet 9c19dc00e54743029dfe71fab44942f1 from peer d5ab76853ad3402bbfd599a73683f7fe at {username='slave'} at 127.0.0.1:59982: session id = d5ab76853ad3402bbfd599a73683f7fe-9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:53.607900 18687 tablet_copy_source_session.cc:215] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:57:53.608690 19375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c19dc00e54743029dfe71fab44942f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:57:53.610495 19375 tablet_copy_client.cc:806] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: tablet copy: Starting download of 0 data blocks...
I20260811 01:57:53.610639 19375 tablet_copy_client.cc:670] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: tablet copy: Starting download of 1 WAL segments...
I20260811 01:57:53.611644 19375 tablet_copy_client.cc:538] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:57:53.612716 19375 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Bootstrap starting.
I20260811 01:57:53.616197 19375 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:57:53.616407 19375 tablet_bootstrap.cc:492] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Bootstrap complete.
I20260811 01:57:53.616556 19375 ts_tablet_manager.cc:1403] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260811 01:57:53.616961 19375 raft_consensus.cc:359] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } }
I20260811 01:57:53.617055 19375 raft_consensus.cc:740] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: d5ab76853ad3402bbfd599a73683f7fe, State: Initialized, Role: LEARNER
I20260811 01:57:53.617151 19375 consensus_queue.cc:260] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: NON_VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: true } }
I20260811 01:57:53.617465 19375 ts_tablet_manager.cc:1434] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 01:57:53.617921 18687 tablet_copy_service.cc:342] P 3491269ab9ef47d4804468f6f50a43fc: Request end of tablet copy session d5ab76853ad3402bbfd599a73683f7fe-9c19dc00e54743029dfe71fab44942f1 received from {username='slave'} at 127.0.0.1:59982
I20260811 01:57:53.618005 18687 tablet_copy_service.cc:434] P 3491269ab9ef47d4804468f6f50a43fc: ending tablet copy session d5ab76853ad3402bbfd599a73683f7fe-9c19dc00e54743029dfe71fab44942f1 on tablet 9c19dc00e54743029dfe71fab44942f1 with peer d5ab76853ad3402bbfd599a73683f7fe
I20260811 01:57:53.720079 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:53.720270 18657 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:53.720304 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:53.720381 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 1 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 9c19dc00e54743029dfe71fab44942f1 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
142af6a7a51b465882ddef279ba1b17e (127.10.166.132:42437): RUNNING
3491269ab9ef47d4804468f6f50a43fc (127.10.166.130:35045): RUNNING [LEADER]
d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129:32835): RUNNING [NONVOTER]
All reported replicas are:
A = 142af6a7a51b465882ddef279ba1b17e
B = 3491269ab9ef47d4804468f6f50a43fc
C = d5ab76853ad3402bbfd599a73683f7fe
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C~ | | | Yes
A | A B* C~ | 2 | 4 | Yes
B | A B* C~ | 2 | 4 | Yes
C | A B C~ | 2 | 4 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260811 01:57:53.747097 10906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260811 01:57:54.145437 18548 raft_consensus.cc:1217] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 LEARNER]: Deduplicated request from leader. Original: 2.3->[2.4-2.4] Dedup: 2.4->[]
I20260811 01:57:54.583412 19373 raft_consensus.cc:1064] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: attempting to promote NON_VOTER d5ab76853ad3402bbfd599a73683f7fe to VOTER
I20260811 01:57:54.583832 19373 consensus_queue.cc:237] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false } }
I20260811 01:57:54.584985 18548 raft_consensus.cc:1275] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 LEARNER]: Refusing update from remote peer 3491269ab9ef47d4804468f6f50a43fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260811 01:57:54.585017 18916 raft_consensus.cc:1275] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Refusing update from remote peer 3491269ab9ef47d4804468f6f50a43fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260811 01:57:54.585386 19431 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:54.585597 19432 consensus_queue.cc:1048] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:57:54.587177 19432 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false } } }
I20260811 01:57:54.587600 18916 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false } } }
I20260811 01:57:54.587858 18548 raft_consensus.cc:2955] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false } } }
I20260811 01:57:54.590003 18378 catalog_manager.cc:5719] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc reported cstate change: config changed from index 4 to 5, d5ab76853ad3402bbfd599a73683f7fe (127.10.166.129) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3491269ab9ef47d4804468f6f50a43fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "142af6a7a51b465882ddef279ba1b17e" member_type: VOTER last_known_addr { host: "127.10.166.132" port: 42437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3491269ab9ef47d4804468f6f50a43fc" member_type: VOTER last_known_addr { host: "127.10.166.130" port: 35045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5ab76853ad3402bbfd599a73683f7fe" member_type: VOTER last_known_addr { host: "127.10.166.129" port: 32835 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260811 01:57:54.791666 18900 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:54.792011 18657 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260811 01:57:54.792639 18783 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260811 01:57:54.793952 18533 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
2771ba0f778f4ff8ab7b279a7be78839 | 127.10.166.190:36097 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
142af6a7a51b465882ddef279ba1b17e | 127.10.166.132:42437 | HEALTHY | <none> | 0 | 0
3491269ab9ef47d4804468f6f50a43fc | 127.10.166.130:35045 | HEALTHY | <none> | 1 | 0
b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | HEALTHY | <none> | 0 | 0
d5ab76853ad3402bbfd599a73683f7fe | 127.10.166.129:32835 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+----------------------+---------------
Small | b79b8a503e864a07ae4c670f5e41d9f2 | 127.10.166.131:40679 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260811 01:57:54.819178 10906 log_verifier.cc:126] Checking tablet 9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:54.837023 10906 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 9c19dc00e54743029dfe71fab44942f1
I20260811 01:57:54.838228 10906 tablet_server.cc:179] TabletServer@127.10.166.129:0 shutting down...
I20260811 01:57:54.842260 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:54.842461 10906 tablet_replica.cc:333] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe: stopping tablet replica
I20260811 01:57:54.842567 10906 raft_consensus.cc:2243] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:54.842698 10906 raft_consensus.cc:2272] T 9c19dc00e54743029dfe71fab44942f1 P d5ab76853ad3402bbfd599a73683f7fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:54.856349 10906 tablet_server.cc:196] TabletServer@127.10.166.129:0 shutdown complete.
I20260811 01:57:54.858355 10906 tablet_server.cc:179] TabletServer@127.10.166.130:0 shutting down...
I20260811 01:57:54.862741 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:54.862921 10906 tablet_replica.cc:333] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc: stopping tablet replica
I20260811 01:57:54.863031 10906 raft_consensus.cc:2243] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 LEADER]: Raft consensus shutting down.
I20260811 01:57:54.863227 10906 raft_consensus.cc:2272] T 9c19dc00e54743029dfe71fab44942f1 P 3491269ab9ef47d4804468f6f50a43fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:54.866969 10906 tablet_server.cc:196] TabletServer@127.10.166.130:0 shutdown complete.
I20260811 01:57:54.868738 10906 tablet_server.cc:179] TabletServer@127.10.166.131:0 shutting down...
I20260811 01:57:54.872629 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:54.884632 10906 tablet_server.cc:196] TabletServer@127.10.166.131:0 shutdown complete.
I20260811 01:57:54.886276 10906 tablet_server.cc:179] TabletServer@127.10.166.132:0 shutting down...
I20260811 01:57:54.890316 10906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260811 01:57:54.890494 10906 tablet_replica.cc:333] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e: stopping tablet replica
I20260811 01:57:54.890599 10906 raft_consensus.cc:2243] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 01:57:54.890731 10906 raft_consensus.cc:2272] T 9c19dc00e54743029dfe71fab44942f1 P 142af6a7a51b465882ddef279ba1b17e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:54.904932 10906 tablet_server.cc:196] TabletServer@127.10.166.132:0 shutdown complete.
I20260811 01:57:54.906591 10906 master.cc:562] Master@127.10.166.190:36097 shutting down...
I20260811 01:57:54.910037 10906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:57:54.910154 10906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:57:54.910197 10906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2771ba0f778f4ff8ab7b279a7be78839: stopping tablet replica
I20260811 01:57:54.922551 10906 master.cc:584] Master@127.10.166.190:36097 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (4501 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (9895 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (48478 ms total)
[ PASSED ] 12 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260811 01:57:54.928725 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~10 seconds ago
I20260811 01:57:54.928769 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 1 messages since previous log ~10 seconds ago
I20260811 01:57:54.928791 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 1 messages since previous log ~10 seconds ago
I20260811 01:57:54.928813 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 17 messages since previous log ~14 seconds ago
I20260811 01:57:54.928833 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 431 messages since previous log ~33 seconds ago
I20260811 01:57:54.928854 10906 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 11 messages since previous log ~16 seconds ago