TsTabletManagerITest.TestTableStats: /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:906: Failure
Failed
Bad status: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260925 13:11:56.621472   392 tablet_server.cc:178] TabletServer@127.0.98.1:43385 shutting down...
I20260925 13:11:56.622201  2500 tablet_copy_service.cc:140] P 0b2184873e294eb78e5d9ad4f879fe03: Received BeginTabletCopySession request for tablet 077b55ad10884161a8f316758fc55be1 from peer a75e127731684428a5e320ef5a358a3e ({username='slave'} at 127.0.0.1:50010)
I20260925 13:11:56.622287  2500 tablet_copy_service.cc:161] P 0b2184873e294eb78e5d9ad4f879fe03: Beginning new tablet copy session on tablet 077b55ad10884161a8f316758fc55be1 from peer a75e127731684428a5e320ef5a358a3e at {username='slave'} at 127.0.0.1:50010: session id = a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1
I20260925 13:11:56.623278  2500 tablet_copy_source_session.cc:205] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:11:56.623942  3074 ts_tablet_manager.cc:1905] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260925 13:11:56.624895   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.625068   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625173   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.625233   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.625543   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625583   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.625607   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.625839   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625875   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.625926   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.626183  3074 ts_tablet_manager.cc:1918] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 3.4
I20260925 13:11:56.626295  3074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077b55ad10884161a8f316758fc55be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.627189  3074 tablet_copy_client.cc:791] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Starting download of 0 data blocks...
I20260925 13:11:56.627372  3074 tablet_copy_client.cc:655] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Starting download of 1 WAL segments...
I20260925 13:11:56.628625  3074 tablet_copy_client.cc:531] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:11:56.629509  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
W20260925 13:11:56.630194  2454 consensus_peers.cc:487] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385): Couldn't send request to peer 919cdc37e0f843f7b2dac0b863a27e22. Status: Network error: Client connection negotiation failed: client connection to 127.0.98.1:43385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.631425  3074 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.631477  3074 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.631726  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.631919  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap complete.
I20260925 13:11:56.632573  3074 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.001s
I20260925 13:11:56.632726  3074 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
I20260925 13:11:56.632808  3074 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: LEARNER
I20260925 13:11:56.632831  3074 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
I20260925 13:11:56.632975  3074 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.633312  2500 tablet_copy_service.cc:342] P 0b2184873e294eb78e5d9ad4f879fe03: Request end of tablet copy session a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1 received from {username='slave'} at 127.0.0.1:50010
I20260925 13:11:56.633364  2500 tablet_copy_service.cc:433] P 0b2184873e294eb78e5d9ad4f879fe03: ending tablet copy session a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1 on tablet 077b55ad10884161a8f316758fc55be1 with peer a75e127731684428a5e320ef5a358a3e
I20260925 13:11:56.639925   392 tablet_server.cc:194] TabletServer@127.0.98.1:43385 shutdown complete.
I20260925 13:11:56.641166   392 tablet_server.cc:178] TabletServer@127.0.98.2:40761 shutting down...
I20260925 13:11:56.644915   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.645578   392 tablet_replica.cc:330] stopping tablet replica
W20260925 13:11:56.645608  2454 consensus_peers.cc:487] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): Couldn't send request to peer a75e127731684428a5e320ef5a358a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.645677   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Raft consensus shutting down.
I20260925 13:11:56.645725   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Raft consensus is shut down!
I20260925 13:11:56.646036   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.646080   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.646108   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.646356   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.646396   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.646451   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.659016   392 tablet_server.cc:194] TabletServer@127.0.98.2:40761 shutdown complete.
I20260925 13:11:56.660346   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:11:56.663287   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.663407   392 tablet_replica.cc:330] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.663471   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.663578   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.663848   392 tablet_replica.cc:330] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.663892   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.663924   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.664131   392 tablet_replica.cc:330] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.664168   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.664218   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.676481   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:11:56.677732   392 master.cc:537] Master@127.0.98.62:40169 shutting down...
I20260925 13:11:56.680263   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.680326   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.680374   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: stopping tablet replica
W20260925 13:11:56.682209   392 rolling_log.cc:182] Unable to compress old log file: Not found: Unable to open input file to compress: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/ts_tablet_manager-itest.dist-test-slave-2538.slave.diagnostics.20260925-131152.0.392: No such file or directory (error 2)
I20260925 13:11:56.682346   392 master.cc:552] Master@127.0.98.62:40169 shutdown complete.
I20260925 13:11:56.683591   392 master.cc:537] Master@127.0.98.61:46353 shutting down...
I20260925 13:11:56.686067   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.686133   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.686151   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.698205   392 master.cc:552] Master@127.0.98.61:46353 shutdown complete.
I20260925 13:11:56.699549   392 master.cc:537] Master@127.0.98.60:34327 shutting down...
I20260925 13:11:56.701961   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.702055   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.702095   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.703748   392 master.cc:552] Master@127.0.98.60:34327 shutdown complete.
I20260925 13:11:56.705485   392 test_util.cc:175] -----------------------------------------------
I20260925 13:11:56.705570   392 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0

[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN      ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260925 13:11:16.782881   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:33221
I20260925 13:11:16.783304   392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:16.783507   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:16.787344   392 server_base.cc:1034] running on GCE node
W20260925 13:11:16.787343   402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.787344   404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.787410   401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.788043   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:16.788100   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:16.788125   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341876788124 us; error 0 us; skew 500 ppm
I20260925 13:11:16.795473   392 webserver.cc:460] Webserver started at http://127.0.98.62:43287/ using document root <none> and password file <none>
I20260925 13:11:16.795754   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:16.795819   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:16.795980   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:16.797544   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "11e0c84710254d69b7aa03ce88923ec6"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.799239   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:16.800073   409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.800311   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:16.800457   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "11e0c84710254d69b7aa03ce88923ec6"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.800558   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:16.807672   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:16.808012   392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:16.808175   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:16.812956   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:33221
I20260925 13:11:16.812994   461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:33221 every 8 connection(s)
I20260925 13:11:16.813861   462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.817023   462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: Bootstrap starting.
I20260925 13:11:16.818033   462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.818341   462 log.cc:824] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:16.819063   462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: No bootstrap required, opened a new log
I20260925 13:11:16.820564   462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER }
I20260925 13:11:16.820691   462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:16.820714   462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e0c84710254d69b7aa03ce88923ec6, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.820930   462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER }
I20260925 13:11:16.821012   462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:16.821064   462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:16.821138   462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:16.821682   462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER }
I20260925 13:11:16.821774   462 leader_election.cc:302] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11e0c84710254d69b7aa03ce88923ec6; no voters: 
I20260925 13:11:16.822002   462 leader_election.cc:288] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:16.822115   465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:16.822302   462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:16.822315   465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 1 LEADER]: Becoming Leader. State: Replica: 11e0c84710254d69b7aa03ce88923ec6, State: Running, Role: LEADER
I20260925 13:11:16.822497   465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER }
I20260925 13:11:16.822897   467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e0c84710254d69b7aa03ce88923ec6. Latest consensus state: current_term: 1 leader_uuid: "11e0c84710254d69b7aa03ce88923ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER } }
I20260925 13:11:16.822897   466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11e0c84710254d69b7aa03ce88923ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0c84710254d69b7aa03ce88923ec6" member_type: VOTER } }
I20260925 13:11:16.823092   466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:16.823321   474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:16.823339   467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:16.823881   474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:16.824023   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:16.825572   474 catalog_manager.cc:1347] Generated new cluster ID: 475cfdc70e784be989138d10c7e36156
I20260925 13:11:16.825629   474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:16.836949   474 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:16.837659   474 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:16.844198   474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: Generated new TSK 0
I20260925 13:11:16.844580   474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:16.856206   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:16.858270   485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.858506   484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.858616   392 server_base.cc:1034] running on GCE node
W20260925 13:11:16.858321   487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.858820   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:16.858868   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:16.858888   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341876858888 us; error 0 us; skew 500 ppm
I20260925 13:11:16.859777   392 webserver.cc:460] Webserver started at http://127.0.98.1:33961/ using document root <none> and password file <none>
I20260925 13:11:16.859870   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:16.859915   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:16.859962   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:16.860280   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "33ce28582b6e4d658018a3a4dac9faaf"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.861512   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:16.862141   492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.862324   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:16.862385   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "33ce28582b6e4d658018a3a4dac9faaf"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.862429   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:16.874887   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:16.875401   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:16.875625   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:16.876016   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:16.876093   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.876125   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:16.876144   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.881026   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:45943
I20260925 13:11:16.881136   555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:45943 every 8 connection(s)
I20260925 13:11:16.881547   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:16.883090   558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.886049   559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.886204   562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.886680   392 server_base.cc:1034] running on GCE node
I20260925 13:11:16.886804   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:16.886842   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:16.886857   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341876886857 us; error 0 us; skew 500 ppm
I20260925 13:11:16.887427   392 webserver.cc:460] Webserver started at http://127.0.98.2:36143/ using document root <none> and password file <none>
I20260925 13:11:16.887540   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:16.887616   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:16.887676   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:16.887985   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "621d498d762a46bea6594d93844839f5"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.888813   556 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33221
I20260925 13:11:16.888908   556 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:16.889052   556 heartbeater.cc:511] Master 127.0.98.62:33221 requested a full tablet report, sending...
I20260925 13:11:16.889468   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:16.889889   424 ts_manager.cc:194] Registered new tserver with Master: 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1:45943)
I20260925 13:11:16.890087   568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.890198   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.890234   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "621d498d762a46bea6594d93844839f5"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-2538"
I20260925 13:11:16.890278   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:16.890761   424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48938
I20260925 13:11:16.930851   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:16.931167   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:16.931337   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:16.931595   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:16.931641   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.931674   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:16.931691   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.936362   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:42843
I20260925 13:11:16.936515   631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:42843 every 8 connection(s)
I20260925 13:11:16.940343   632 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33221
I20260925 13:11:16.940438   632 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:16.940513   632 heartbeater.cc:511] Master 127.0.98.62:33221 requested a full tablet report, sending...
I20260925 13:11:16.940758   424 ts_manager.cc:194] Registered new tserver with Master: 621d498d762a46bea6594d93844839f5 (127.0.98.2:42843)
I20260925 13:11:16.941190   424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48954
I20260925 13:11:16.941130   392 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.004507783s
I20260925 13:11:16.948199   424 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48962:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  hash_schema {
    columns {
      name: "key"
    }
    num_buckets: 2
    seed: 0
  }
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260925 13:11:16.949951   424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260925 13:11:16.956471   596 tablet_service.cc:1467] Processing CreateTablet for tablet 4d81e496939b481b9815ae0d7bf82421 (DEFAULT_TABLE table=test-table [id=43935347b563490591fe8d4159adbe77]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:16.956629   520 tablet_service.cc:1467] Processing CreateTablet for tablet 4d81e496939b481b9815ae0d7bf82421 (DEFAULT_TABLE table=test-table [id=43935347b563490591fe8d4159adbe77]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:16.956624   519 tablet_service.cc:1467] Processing CreateTablet for tablet 8666970b92f94dd1bd3b85aed5bf7745 (DEFAULT_TABLE table=test-table [id=43935347b563490591fe8d4159adbe77]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:16.956710   596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d81e496939b481b9815ae0d7bf82421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.956739   520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d81e496939b481b9815ae0d7bf82421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.956794   519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8666970b92f94dd1bd3b85aed5bf7745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.958474   595 tablet_service.cc:1467] Processing CreateTablet for tablet 8666970b92f94dd1bd3b85aed5bf7745 (DEFAULT_TABLE table=test-table [id=43935347b563490591fe8d4159adbe77]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:16.958583   595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8666970b92f94dd1bd3b85aed5bf7745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.958760   645 tablet_bootstrap.cc:492] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: Bootstrap starting.
I20260925 13:11:16.959527   645 tablet_bootstrap.cc:654] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.959991   646 tablet_bootstrap.cc:492] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: Bootstrap starting.
I20260925 13:11:16.960219   645 tablet_bootstrap.cc:492] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: No bootstrap required, opened a new log
I20260925 13:11:16.960284   645 ts_tablet_manager.cc:1397] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:16.960472   646 tablet_bootstrap.cc:654] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.960619   645 raft_consensus.cc:348] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.960721   645 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.960777   645 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.960984   645 ts_tablet_manager.cc:1428] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:16.961145   645 tablet_bootstrap.cc:492] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: Bootstrap starting.
I20260925 13:11:16.961143   632 heartbeater.cc:503] Master 127.0.98.62:33221 was elected leader, sending a full tablet report...
I20260925 13:11:16.961145   646 tablet_bootstrap.cc:492] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: No bootstrap required, opened a new log
I20260925 13:11:16.961352   646 ts_tablet_manager.cc:1397] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:16.961514   646 raft_consensus.cc:348] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.961575   646 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.961604   646 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.961760   646 ts_tablet_manager.cc:1428] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.961819   556 heartbeater.cc:503] Master 127.0.98.62:33221 was elected leader, sending a full tablet report...
I20260925 13:11:16.961972   646 tablet_bootstrap.cc:492] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: Bootstrap starting.
I20260925 13:11:16.962349   645 tablet_bootstrap.cc:654] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.962540   646 tablet_bootstrap.cc:654] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.962910   645 tablet_bootstrap.cc:492] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: No bootstrap required, opened a new log
I20260925 13:11:16.962954   645 ts_tablet_manager.cc:1397] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:16.963133   645 raft_consensus.cc:348] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.963189   645 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.963222   645 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.963301   645 ts_tablet_manager.cc:1428] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.963613   646 tablet_bootstrap.cc:492] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: No bootstrap required, opened a new log
I20260925 13:11:16.963657   646 ts_tablet_manager.cc:1397] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:16.963797   646 raft_consensus.cc:348] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.963846   646 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.963872   646 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.963943   646 ts_tablet_manager.cc:1428] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:16.971388   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:16.971487   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Emulating election...
I20260925 13:11:16.971516   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:16.972103   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:16.972182   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:16.972517   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:16.973037   424 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 1 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:16.979826   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:17.080832   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
W20260925 13:11:17.131593   557 tablet.cc:2369] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:17.282126   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:17.467864   530 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:17.469151   649 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:17.583053   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:17.583153   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Emulating election...
I20260925 13:11:17.583182   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:17.583828   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:17.583917   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:17.584175   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:17.584429   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 1 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:17.584916   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:17.686128   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:17.887544   392 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:17.965432   530 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:17.966560   649 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:18.388813   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:18.388921   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Emulating election...
I20260925 13:11:18.388957   392 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:18.388976   392 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:18.389031   392 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:18.389288   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:18.390003   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 2 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:18.390087   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:18.390276   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:18.390637   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:18.390998   392 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260925 13:11:18.491863   392 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260925 13:11:18.693109   392 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260925 13:11:18.833232   530 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:18.833963   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 13:11:18.834306   649 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:18.994287   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:18.994357   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 1 FOLLOWER]: Emulating election...
I20260925 13:11:18.994378   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:18.994998   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:18.995055   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:18.995460   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:18.995805   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 1 to 2, leader changed from 621d498d762a46bea6594d93844839f5 (127.0.98.2) to 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1). New cstate: current_term: 2 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:18.996204   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:19.097604   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:19.298695   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:19.540904   606 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:19.541018   606 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:19.541061   606 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:19.541193   606 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:19.542007   606 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 2 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 13:11:19.542315   678 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:19.800135   392 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:19.800215   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 FOLLOWER]: Emulating election...
I20260925 13:11:19.800238   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:19.800992   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:19.801162   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:19.801383   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:19.801793   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 2 to 3, leader changed from 621d498d762a46bea6594d93844839f5 (127.0.98.2) to 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1). New cstate: current_term: 3 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:19.802086   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:19.903226   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:19.923832   530 raft_consensus.cc:1226] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 LEADER]: Rejecting Update request from peer 621d498d762a46bea6594d93844839f5 for earlier term 2. Current term is 3. Ops: []
I20260925 13:11:19.924216   649 consensus_queue.cc:1046] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:19.924475   649 consensus_queue.cc:1225] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer 33ce28582b6e4d658018a3a4dac9faaf log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260925 13:11:19.924535   688 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:19.924569   688 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:19.924610   688 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:19.924680   688 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:20.104646   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:20.282812   606 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 3 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260925 13:11:20.283258   682 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:20.405871   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:20.405956   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 3 FOLLOWER]: Emulating election...
I20260925 13:11:20.405977   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:20.406726   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:20.406788   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:20.406958   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:20.407274   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 3 to 4, leader changed from 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1) to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 4 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:20.407739   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:20.508790   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:20.699620   606 raft_consensus.cc:1226] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Rejecting Update request from peer 33ce28582b6e4d658018a3a4dac9faaf for earlier term 3. Current term is 4. Ops: []
I20260925 13:11:20.699977   690 consensus_queue.cc:1046] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:20.700124   690 consensus_queue.cc:1225] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer 621d498d762a46bea6594d93844839f5 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260925 13:11:20.700189   682 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 LEADER]: Stepping down as leader of term 3
I20260925 13:11:20.700250   682 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:20.700297   682 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:20.700403   682 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:20.710016   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:20.884415   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 4 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260925 13:11:20.884821   688 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:21.211354   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:21.211438   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Emulating election...
I20260925 13:11:21.211462   392 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Stepping down as leader of term 4
I20260925 13:11:21.211475   392 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:21.211568   392 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:21.211680   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:21.262678   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 5 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:21.262790   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:21.262998   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:21.263425   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 4 to 5. New cstate: current_term: 5 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:21.263739   392 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:21.365167   392 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:21.566419   392 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:21.778515   530 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:21.779415   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260925 13:11:21.779760   703 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:21.867846   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:21.867925   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 2 FOLLOWER]: Emulating election...
I20260925 13:11:21.867985   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:21.868767   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:21.868855   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:21.869191   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:21.869488   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 2 to 3, leader changed from 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1) to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 3 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:21.869876   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 3.3
I20260925 13:11:21.910341   606 raft_consensus.cc:1226] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 LEADER]: Rejecting Update request from peer 33ce28582b6e4d658018a3a4dac9faaf for earlier term 2. Current term is 3. Ops: []
I20260925 13:11:21.910667   702 consensus_queue.cc:1046] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:21.911033   702 consensus_queue.cc:1225] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer 621d498d762a46bea6594d93844839f5 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260925 13:11:21.911085   711 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:21.911163   711 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:21.911218   711 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:21.911285   711 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:21.971132   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 3.3
I20260925 13:11:22.172228   392 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 3.3
I20260925 13:11:22.459133   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260925 13:11:22.459558   704 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:22.673393   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:22.673516   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 LEADER]: Emulating election...
I20260925 13:11:22.673552   392 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 LEADER]: Stepping down as leader of term 3
I20260925 13:11:22.673573   392 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:22.673631   392 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:22.673770   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:22.674749   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:22.674835   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:22.675007   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:22.675321   424 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:22.675650   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:22.776918   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:22.978017   392 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:23.183319   530 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:23.184113   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 4 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260925 13:11:23.184413   714 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:23.279539   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:23.279652   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 4 FOLLOWER]: Emulating election...
I20260925 13:11:23.279688   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:23.280478   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:23.280561   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:23.280743   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:23.281005   424 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 4 to 5, leader changed from 621d498d762a46bea6594d93844839f5 (127.0.98.2) to 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1). New cstate: current_term: 5 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:23.281368   392 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 4.4
I20260925 13:11:23.382627   392 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 4.4
I20260925 13:11:23.584020   392 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 4.4
I20260925 13:11:23.684110   530 raft_consensus.cc:1226] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 LEADER]: Rejecting Update request from peer 621d498d762a46bea6594d93844839f5 for earlier term 4. Current term is 5. Ops: []
I20260925 13:11:23.684454   714 consensus_queue.cc:1046] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:23.684598   714 consensus_queue.cc:1225] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer 33ce28582b6e4d658018a3a4dac9faaf log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260925 13:11:23.684638   721 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Stepping down as leader of term 4
I20260925 13:11:23.684698   721 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:23.684731   721 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:23.684803   721 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:23.885188   392 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 4.4
I20260925 13:11:23.890797   606 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 5 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260925 13:11:23.891120   727 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:24.486811   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:24.486897   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 LEADER]: Emulating election...
I20260925 13:11:24.486922   392 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 LEADER]: Stepping down as leader of term 5
I20260925 13:11:24.486935   392 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:24.487013   392 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:24.487147   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:24.487941   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:24.488027   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:24.488188   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:24.488524   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 5 to 6. New cstate: current_term: 6 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:24.488926   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:24.590135   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:24.791240   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:24.938596   606 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:24.939371   606 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260925 13:11:24.939697   731 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:25.092577   392 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:25.092669   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 FOLLOWER]: Emulating election...
I20260925 13:11:25.092705   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:25.093429   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:25.093487   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:25.093851   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:25.094337   424 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 5 to 6, leader changed from 621d498d762a46bea6594d93844839f5 (127.0.98.2) to 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1). New cstate: current_term: 6 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:25.094839   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:25.195843   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:25.235224   530 raft_consensus.cc:1226] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Rejecting Update request from peer 621d498d762a46bea6594d93844839f5 for earlier term 5. Current term is 6. Ops: []
I20260925 13:11:25.235535   721 consensus_queue.cc:1046] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:25.235796   721 consensus_queue.cc:1225] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer 33ce28582b6e4d658018a3a4dac9faaf log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260925 13:11:25.235847   744 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 5 LEADER]: Stepping down as leader of term 5
I20260925 13:11:25.235898   744 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:25.235968   744 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:25.236090   744 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:25.397043   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 5.5
I20260925 13:11:25.533130   606 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260925 13:11:25.533442   737 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:25.898437   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:25.898598   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Emulating election...
I20260925 13:11:25.898628   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:25.901180   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:25.901309   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:25.901635   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:25.901973   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 6 to 7, leader changed from 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1) to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 7 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:25.902499   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.003786   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.039429   606 raft_consensus.cc:1226] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Rejecting Update request from peer 33ce28582b6e4d658018a3a4dac9faaf for earlier term 6. Current term is 7. Ops: []
I20260925 13:11:26.039794   752 consensus_queue.cc:1046] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:26.039995   752 consensus_queue.cc:1225] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer 621d498d762a46bea6594d93844839f5 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260925 13:11:26.040025   752 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Stepping down as leader of term 6
I20260925 13:11:26.040042   752 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:26.040067   752 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:26.040149   752 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:26.205011   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.346985   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 7 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260925 13:11:26.347460   744 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:26.506111   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:26.506192   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Emulating election...
I20260925 13:11:26.506213   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:26.506824   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:26.506881   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:26.507057   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:26.507375   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 6 to 7, leader changed from 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1) to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 7 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:26.507890   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.568066   606 raft_consensus.cc:1226] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Rejecting Update request from peer 33ce28582b6e4d658018a3a4dac9faaf for earlier term 6. Current term is 7. Ops: []
I20260925 13:11:26.568351   752 consensus_queue.cc:1046] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:26.568514   752 consensus_queue.cc:1225] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer 621d498d762a46bea6594d93844839f5 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260925 13:11:26.568573   752 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Stepping down as leader of term 6
I20260925 13:11:26.568599   752 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:26.568632   752 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:26.568703   752 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:26.609167   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.810513   392 cluster_itest_util.cc:256] Not converged past 0 yet: 6.6 7.7
I20260925 13:11:26.906338   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 7 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260925 13:11:26.906618   744 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:27.311527   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:27.311645   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Emulating election...
I20260925 13:11:27.311676   392 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Stepping down as leader of term 7
I20260925 13:11:27.311695   392 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:27.311722   392 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:27.311801   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:27.312474   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:27.312528   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:27.312672   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:27.312978   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 7 to 8. New cstate: current_term: 8 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:27.313413   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:27.414280   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:27.615592   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:27.858035   530 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:27.858820   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 8 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260925 13:11:27.859149   769 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:11:27.916818   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:27.916916   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Emulating election...
I20260925 13:11:27.916950   392 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Stepping down as leader of term 7
I20260925 13:11:27.916975   392 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:27.917026   392 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:27.917189   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:27.917917   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:27.918007   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:27.918291   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:27.918643   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 7 to 8. New cstate: current_term: 8 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:27.918980   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:28.020593   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:28.221717   392 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 8.8
I20260925 13:11:28.372704   530 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:28.373564   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 8 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260925 13:11:28.373890   771 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:11:28.723271   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:28.723430   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Emulating election...
I20260925 13:11:28.723472   392 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Stepping down as leader of term 8
I20260925 13:11:28.723512   392 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:28.723578   392 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:28.723759   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:28.724622   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 9 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:28.724723   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:28.724941   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:28.725339   426 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 8 to 9. New cstate: current_term: 9 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:28.725801   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:28.827033   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:29.028527   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:29.191286   530 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:29.192094   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260925 13:11:29.192483   780 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:11:29.329771   392 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260925 13:11:29.329887   392 raft_consensus.cc:412] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Emulating election...
I20260925 13:11:29.329919   392 raft_consensus.cc:3032] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Stepping down as leader of term 8
I20260925 13:11:29.329941   392 raft_consensus.cc:729] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:29.329998   392 consensus_queue.cc:260] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:29.330152   392 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:29.330899   392 raft_consensus.cc:686] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 9 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:29.330969   392 consensus_queue.cc:237] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:29.331202   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:29.331501   426 catalog_manager.cc:5495] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 8 to 9. New cstate: current_term: 9 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:29.331771   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:29.432976   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:29.634291   392 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 9.9
I20260925 13:11:29.740613   530 raft_consensus.cc:3037] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:29.741573   530 raft_consensus.cc:1261] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260925 13:11:29.741931   780 consensus_queue.cc:1035] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:11:30.136817   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:30.136941   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Emulating election...
I20260925 13:11:30.136977   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:30.137724   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 10 LEADER]: Becoming Leader. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:30.137955   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:30.138300   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:30.138722   425 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf reported cstate change: term changed from 9 to 10, leader changed from 621d498d762a46bea6594d93844839f5 (127.0.98.2) to 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1). New cstate: current_term: 10 leader_uuid: "33ce28582b6e4d658018a3a4dac9faaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:30.139110   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 9.9
I20260925 13:11:30.147043   530 raft_consensus.cc:1226] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 10 LEADER]: Rejecting Update request from peer 621d498d762a46bea6594d93844839f5 for earlier term 9. Current term is 10. Ops: []
I20260925 13:11:30.147480   796 consensus_queue.cc:1046] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:11:30.147619   796 consensus_queue.cc:1225] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Peer 33ce28582b6e4d658018a3a4dac9faaf log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260925 13:11:30.147653   780 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 9 LEADER]: Stepping down as leader of term 9
I20260925 13:11:30.147696   780 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:30.147727   780 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:30.147830   780 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:30.240113   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 9.9
I20260925 13:11:30.441475   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 9.9
I20260925 13:11:30.603674   606 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 10 FOLLOWER]: Refusing update from remote peer 33ce28582b6e4d658018a3a4dac9faaf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260925 13:11:30.604043   794 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:11:30.742523   392 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260925 13:11:30.742617   392 raft_consensus.cc:412] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 10 FOLLOWER]: Emulating election...
I20260925 13:11:30.742640   392 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 10 FOLLOWER]: Advancing to term 11
I20260925 13:11:30.743377   392 raft_consensus.cc:686] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 11 LEADER]: Becoming Leader. State: Replica: 621d498d762a46bea6594d93844839f5, State: Running, Role: LEADER
I20260925 13:11:30.743479   392 consensus_queue.cc:237] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:30.743643   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:30.743907   425 catalog_manager.cc:5495] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 reported cstate change: term changed from 10 to 11, leader changed from 33ce28582b6e4d658018a3a4dac9faaf (127.0.98.1) to 621d498d762a46bea6594d93844839f5 (127.0.98.2). New cstate: current_term: 11 leader_uuid: "621d498d762a46bea6594d93844839f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:30.744283   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 11.11
I20260925 13:11:30.845404   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 11.11
I20260925 13:11:31.046608   392 cluster_itest_util.cc:256] Not converged past 0 yet: 10.10 11.11
I20260925 13:11:31.102880   606 raft_consensus.cc:1226] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 11 LEADER]: Rejecting Update request from peer 33ce28582b6e4d658018a3a4dac9faaf for earlier term 10. Current term is 11. Ops: []
I20260925 13:11:31.103413   794 consensus_queue.cc:1046] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 13:11:31.103638   794 consensus_queue.cc:1225] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [LEADER]: Peer 621d498d762a46bea6594d93844839f5 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260925 13:11:31.103693   803 raft_consensus.cc:3032] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 10 LEADER]: Stepping down as leader of term 10
I20260925 13:11:31.103732   803 raft_consensus.cc:729] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33ce28582b6e4d658018a3a4dac9faaf, State: Running, Role: LEADER
I20260925 13:11:31.103765   803 consensus_queue.cc:260] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 } } peers { permanent_uuid: "621d498d762a46bea6594d93844839f5" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42843 } }
I20260925 13:11:31.103832   803 raft_consensus.cc:3037] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 10 FOLLOWER]: Advancing to term 11
I20260925 13:11:31.282548   530 raft_consensus.cc:1261] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 11 FOLLOWER]: Refusing update from remote peer 621d498d762a46bea6594d93844839f5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260925 13:11:31.282903   780 consensus_queue.cc:1035] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce28582b6e4d658018a3a4dac9faaf" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 13:11:31.548532   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:31.552028   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:31.552170   392 tablet_replica.cc:330] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf: stopping tablet replica
I20260925 13:11:31.552284   392 raft_consensus.cc:2229] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 11 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:31.552349   392 raft_consensus.cc:2258] T 8666970b92f94dd1bd3b85aed5bf7745 P 33ce28582b6e4d658018a3a4dac9faaf [term 11 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:31.552668   392 tablet_replica.cc:330] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf: stopping tablet replica
I20260925 13:11:31.552708   392 raft_consensus.cc:2229] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:31.552731   392 raft_consensus.cc:2258] T 4d81e496939b481b9815ae0d7bf82421 P 33ce28582b6e4d658018a3a4dac9faaf [term 9 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:31.565951   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:31.567171   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:31.570883   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:31.571120   392 tablet_replica.cc:330] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5: stopping tablet replica
I20260925 13:11:31.571179   392 raft_consensus.cc:2229] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 11 LEADER]: Raft consensus shutting down.
I20260925 13:11:31.571259   392 raft_consensus.cc:2258] T 8666970b92f94dd1bd3b85aed5bf7745 P 621d498d762a46bea6594d93844839f5 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:31.571501   392 tablet_replica.cc:330] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5: stopping tablet replica
I20260925 13:11:31.571535   392 raft_consensus.cc:2229] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 9 LEADER]: Raft consensus shutting down.
I20260925 13:11:31.571565   392 raft_consensus.cc:2258] T 4d81e496939b481b9815ae0d7bf82421 P 621d498d762a46bea6594d93844839f5 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:31.583518   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:31.584523   392 master.cc:537] Master@127.0.98.62:33221 shutting down...
I20260925 13:11:31.587747   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:31.587828   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:31.587872   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 11e0c84710254d69b7aa03ce88923ec6: stopping tablet replica
I20260925 13:11:31.599747   392 master.cc:552] Master@127.0.98.62:33221 shutdown complete.
[       OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (14828 ms)
[ RUN      ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260925 13:11:31.605032   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:34989
I20260925 13:11:31.605268   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:31.607401   817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.607685   816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.607832   819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:31.608219   392 server_base.cc:1034] running on GCE node
I20260925 13:11:31.608315   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:31.608383   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:31.608407   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341891608407 us; error 0 us; skew 500 ppm
I20260925 13:11:31.614244   392 webserver.cc:460] Webserver started at http://127.0.98.62:41141/ using document root <none> and password file <none>
I20260925 13:11:31.614334   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:31.614367   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:31.614404   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:31.614662   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "51f2b60e0c9d45b4b8f680cfc8c380df"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.615649   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:31.616341   824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.616540   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:31.616600   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "51f2b60e0c9d45b4b8f680cfc8c380df"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.616644   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:31.623368   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:31.623557   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:31.626401   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:34989
I20260925 13:11:31.628288   876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:34989 every 8 connection(s)
I20260925 13:11:31.628408   877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:31.629501   877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df: Bootstrap starting.
I20260925 13:11:31.629760   877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:31.630329   877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df: No bootstrap required, opened a new log
I20260925 13:11:31.630440   877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER }
I20260925 13:11:31.630503   877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:31.630518   877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51f2b60e0c9d45b4b8f680cfc8c380df, State: Initialized, Role: FOLLOWER
I20260925 13:11:31.630587   877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER }
I20260925 13:11:31.630640   877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:31.630671   877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:31.630702   877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:31.631186   877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER }
I20260925 13:11:31.631245   877 leader_election.cc:302] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51f2b60e0c9d45b4b8f680cfc8c380df; no voters: 
I20260925 13:11:31.631357   877 leader_election.cc:288] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:31.631404   880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:31.631524   880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 1 LEADER]: Becoming Leader. State: Replica: 51f2b60e0c9d45b4b8f680cfc8c380df, State: Running, Role: LEADER
I20260925 13:11:31.631565   877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:31.631631   880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER }
I20260925 13:11:31.631865   881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER } }
I20260925 13:11:31.631968   881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:31.632061   882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51f2b60e0c9d45b4b8f680cfc8c380df. Latest consensus state: current_term: 1 leader_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f2b60e0c9d45b4b8f680cfc8c380df" member_type: VOTER } }
I20260925 13:11:31.632283   882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:31.632429   890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:31.632614   890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:31.632867   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:31.633384   890 catalog_manager.cc:1347] Generated new cluster ID: d09d348df587453aad7ca2e5fc4072fc
I20260925 13:11:31.633464   890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:31.640564   890 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:31.641043   890 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:31.646994   890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df: Generated new TSK 0
I20260925 13:11:31.647189   890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:31.648674   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:31.650439   898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.650453   901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.650439   899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:31.650707   392 server_base.cc:1034] running on GCE node
I20260925 13:11:31.650816   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:31.650868   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:31.650890   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341891650889 us; error 0 us; skew 500 ppm
I20260925 13:11:31.651755   392 webserver.cc:460] Webserver started at http://127.0.98.1:37501/ using document root <none> and password file <none>
I20260925 13:11:31.651857   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:31.651899   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:31.651978   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:31.652591   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "34e4e0667a4742fa96f84f527c47518f"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.654268   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:31.655148   906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.655287   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:31.655349   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "34e4e0667a4742fa96f84f527c47518f"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.655403   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:31.669946   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:31.670189   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:31.670352   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:31.670565   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:31.670604   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.670655   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:31.670677   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.674136   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:45915
I20260925 13:11:31.674158   969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:45915 every 8 connection(s)
I20260925 13:11:31.674566   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:31.676256   975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.676376   974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:31.676584   392 server_base.cc:1034] running on GCE node
W20260925 13:11:31.676271   977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:31.676808   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:31.676841   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:31.676856   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341891676856 us; error 0 us; skew 500 ppm
I20260925 13:11:31.678021   392 webserver.cc:460] Webserver started at http://127.0.98.2:45699/ using document root <none> and password file <none>
I20260925 13:11:31.678124   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:31.678263   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:31.678329   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:31.678656   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "41497c801aa9425099b05ace25365b8b"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.679965   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20260925 13:11:31.680215   970 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34989
I20260925 13:11:31.680276   970 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:31.680351   970 heartbeater.cc:511] Master 127.0.98.62:34989 requested a full tablet report, sending...
I20260925 13:11:31.680583   841 ts_manager.cc:194] Registered new tserver with Master: 34e4e0667a4742fa96f84f527c47518f (127.0.98.1:45915)
I20260925 13:11:31.680609   982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.680915   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:31.680958   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "41497c801aa9425099b05ace25365b8b"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-2538"
I20260925 13:11:31.681000   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:31.681407   841 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37738
I20260925 13:11:31.699163   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:31.699491   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:31.699703   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:31.699956   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:31.699993   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.700035   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:31.700062   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.703128   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:36717
I20260925 13:11:31.703209  1045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:36717 every 8 connection(s)
I20260925 13:11:31.707293  1046 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34989
I20260925 13:11:31.707373  1046 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:31.707466  1046 heartbeater.cc:511] Master 127.0.98.62:34989 requested a full tablet report, sending...
I20260925 13:11:31.707736   841 ts_manager.cc:194] Registered new tserver with Master: 41497c801aa9425099b05ace25365b8b (127.0.98.2:36717)
I20260925 13:11:31.707738   392 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.004450834s
I20260925 13:11:31.708124   841 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37750
I20260925 13:11:31.713511   841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37754:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260925 13:11:31.713722   841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260925 13:11:31.718664  1010 tablet_service.cc:1467] Processing CreateTablet for tablet 2f21f740158b4b888ed8a92b0c98305d (DEFAULT_TABLE table=test-table [id=4a16833ababa4f4eab60df76129ce9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:31.718845  1010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f21f740158b4b888ed8a92b0c98305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:31.718900   934 tablet_service.cc:1467] Processing CreateTablet for tablet 2f21f740158b4b888ed8a92b0c98305d (DEFAULT_TABLE table=test-table [id=4a16833ababa4f4eab60df76129ce9d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:31.719023   934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f21f740158b4b888ed8a92b0c98305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:31.719996  1059 tablet_bootstrap.cc:492] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: Bootstrap starting.
I20260925 13:11:31.720180  1060 tablet_bootstrap.cc:492] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: Bootstrap starting.
I20260925 13:11:31.720459  1059 tablet_bootstrap.cc:654] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:31.720602  1060 tablet_bootstrap.cc:654] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:31.721266  1059 tablet_bootstrap.cc:492] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: No bootstrap required, opened a new log
I20260925 13:11:31.721307  1059 ts_tablet_manager.cc:1397] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:31.721454  1059 raft_consensus.cc:348] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:31.721511  1059 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Initialized, Role: FOLLOWER
I20260925 13:11:31.721557  1059 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:31.721742  1046 heartbeater.cc:503] Master 127.0.98.62:34989 was elected leader, sending a full tablet report...
I20260925 13:11:31.721743  1059 ts_tablet_manager.cc:1428] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:31.721969  1060 tablet_bootstrap.cc:492] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: No bootstrap required, opened a new log
I20260925 13:11:31.722019  1060 ts_tablet_manager.cc:1397] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:31.722159  1060 raft_consensus.cc:348] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:31.722236  1060 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Initialized, Role: FOLLOWER
I20260925 13:11:31.722263  1060 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:31.722407  1060 ts_tablet_manager.cc:1428] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:31.722465   970 heartbeater.cc:503] Master 127.0.98.62:34989 was elected leader, sending a full tablet report...
I20260925 13:11:31.729791  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:31.729874  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 0 FOLLOWER]: Emulating election...
I20260925 13:11:31.729902  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:31.730584  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 1 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:31.730669  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
W20260925 13:11:31.730731  1046 heartbeater.cc:644] Failed to heartbeat to 127.0.98.62:34989 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260925 13:11:31.730955  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:31.737919  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:31.839092  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:32.040506  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260925 13:11:32.212072   944 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:32.213207  1063 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:32.342900  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:32.342979  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 1 FOLLOWER]: Emulating election...
I20260925 13:11:32.343009  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:32.343775  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:32.343869  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:32.344111  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:32.344791  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:32.445729  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:32.647176  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260925 13:11:32.672636   944 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 LEADER]: Rejecting Update request from peer 41497c801aa9425099b05ace25365b8b for earlier term 1. Current term is 2. Ops: []
I20260925 13:11:32.672967  1063 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:32.673115  1063 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer 34e4e0667a4742fa96f84f527c47518f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260925 13:11:32.673148  1063 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:32.673169  1063 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:32.673199  1063 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:32.673262  1063 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:32.803141  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 2 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 13:11:32.803488  1064 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:32.949434  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:32.949519  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 LEADER]: Emulating election...
I20260925 13:11:32.949542  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:32.949558  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:32.949585  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:32.949688  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:32.950325  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 3 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:32.950424  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:32.950608  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:32.951220  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:33.052503  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:33.253512  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260925 13:11:33.467146  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:33.468204  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 3 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260925 13:11:33.468524  1078 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:33.555889  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:33.555979  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 3 FOLLOWER]: Emulating election...
I20260925 13:11:33.556010  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:33.556933  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:33.557045  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:33.557255  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:33.558004  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:33.659181  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:33.860448  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:33.887698  1020 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 LEADER]: Rejecting Update request from peer 34e4e0667a4742fa96f84f527c47518f for earlier term 3. Current term is 4. Ops: []
I20260925 13:11:33.888134  1078 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:33.888279  1078 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer 41497c801aa9425099b05ace25365b8b log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260925 13:11:33.888366  1080 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 3 LEADER]: Stepping down as leader of term 3
I20260925 13:11:33.888423  1080 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:33.888468  1080 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:33.888564  1080 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:34.161577  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260925 13:11:34.172556   944 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 4 FOLLOWER]: Refusing update from remote peer 41497c801aa9425099b05ace25365b8b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260925 13:11:34.172964  1083 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:34.563923  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:34.564040  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 LEADER]: Emulating election...
I20260925 13:11:34.564074  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 LEADER]: Stepping down as leader of term 4
I20260925 13:11:34.564100  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:34.564157  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:34.564282  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:34.564957  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 5 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:34.565038  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:34.565244  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:34.565986  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:34.667336  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:34.868402  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 4.4 5.5
I20260925 13:11:35.167052   944 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:35.167898   944 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 5 FOLLOWER]: Refusing update from remote peer 41497c801aa9425099b05ace25365b8b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260925 13:11:35.168234  1087 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:35.170951  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:35.171012  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 5 LEADER]: Emulating election...
I20260925 13:11:35.171030  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 5 LEADER]: Stepping down as leader of term 5
I20260925 13:11:35.171043  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:35.171101  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:35.171224  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:35.171869  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 6 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:35.171948  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:35.172079  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:35.172786  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 6.6
I20260925 13:11:35.273852  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 6.6
I20260925 13:11:35.475140  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 5.5 6.6
I20260925 13:11:35.610904   944 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:35.611769   944 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 6 FOLLOWER]: Refusing update from remote peer 41497c801aa9425099b05ace25365b8b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260925 13:11:35.612113  1087 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:35.777540  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:35.777661  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 6 FOLLOWER]: Emulating election...
I20260925 13:11:35.777709  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:35.778486  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:35.778577  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:35.778929  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:35.779867  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 6.6
I20260925 13:11:35.881163  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 6.6
I20260925 13:11:36.082270  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 7.7 6.6
I20260925 13:11:36.136515   944 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 LEADER]: Rejecting Update request from peer 41497c801aa9425099b05ace25365b8b for earlier term 6. Current term is 7. Ops: []
I20260925 13:11:36.136933  1092 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:36.137140  1092 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer 34e4e0667a4742fa96f84f527c47518f log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260925 13:11:36.137220  1093 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 6 LEADER]: Stepping down as leader of term 6
I20260925 13:11:36.137305  1093 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:36.137341  1093 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:36.137467  1093 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:36.234742  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 7 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260925 13:11:36.235095  1088 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:36.384637  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:36.384742  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 LEADER]: Emulating election...
I20260925 13:11:36.384778  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 LEADER]: Stepping down as leader of term 7
I20260925 13:11:36.384819  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:36.384866  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:36.384987  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:36.385779  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 8 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:36.385864  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:36.386030  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:36.386710  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 7.7
I20260925 13:11:36.487696  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 7.7
I20260925 13:11:36.688683  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 8.8 7.7
I20260925 13:11:36.846467  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:36.847272  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 8 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260925 13:11:36.847569  1088 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:11:36.990984  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:36.991106  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 8 LEADER]: Emulating election...
I20260925 13:11:36.991145  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 8 LEADER]: Stepping down as leader of term 8
I20260925 13:11:36.991176  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:36.991228  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:36.991348  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:36.992038  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 9 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:36.992130  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:36.992300  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:36.992985  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 8.8
I20260925 13:11:37.094270  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 8.8
I20260925 13:11:37.295501  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 8.8
I20260925 13:11:37.397149  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:37.398106  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 9 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260925 13:11:37.398424  1088 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:11:37.597884  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:37.597981  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 9 FOLLOWER]: Emulating election...
I20260925 13:11:37.598021  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:37.611853  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 10 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:37.612008  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:37.612272  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:37.612914  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 10.10
I20260925 13:11:37.713936  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 10.10
I20260925 13:11:37.807823  1020 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 10 LEADER]: Rejecting Update request from peer 34e4e0667a4742fa96f84f527c47518f for earlier term 9. Current term is 10. Ops: []
I20260925 13:11:37.808192  1102 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:11:37.808315  1102 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer 41497c801aa9425099b05ace25365b8b log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260925 13:11:37.808377  1102 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 9 LEADER]: Stepping down as leader of term 9
I20260925 13:11:37.808403  1102 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:37.808434  1102 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:37.808499  1102 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:37.914902  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 9.9 10.10
I20260925 13:11:38.056242   944 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 10 FOLLOWER]: Refusing update from remote peer 41497c801aa9425099b05ace25365b8b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260925 13:11:38.056667  1101 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:11:38.217159  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:38.217244  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 10 FOLLOWER]: Emulating election...
I20260925 13:11:38.217265  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 10 FOLLOWER]: Advancing to term 11
I20260925 13:11:38.218132  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:38.218220  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:38.218429  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:38.219231  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 11.11 10.10
I20260925 13:11:38.320343  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 11.11 10.10
I20260925 13:11:38.521574  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 11.11 10.10
I20260925 13:11:38.643558   944 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 LEADER]: Rejecting Update request from peer 41497c801aa9425099b05ace25365b8b for earlier term 10. Current term is 11. Ops: []
I20260925 13:11:38.643913  1108 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 13:11:38.644122  1108 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer 34e4e0667a4742fa96f84f527c47518f log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260925 13:11:38.644176  1108 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 10 LEADER]: Stepping down as leader of term 10
I20260925 13:11:38.644230  1108 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:38.644280  1108 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:38.644469  1108 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 10 FOLLOWER]: Advancing to term 11
I20260925 13:11:38.696326  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 11 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260925 13:11:38.696712  1102 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 13:11:38.823976  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:38.824087  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 LEADER]: Emulating election...
I20260925 13:11:38.824122  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 LEADER]: Stepping down as leader of term 11
I20260925 13:11:38.824142  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:38.824200  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:38.824326  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 11 FOLLOWER]: Advancing to term 12
I20260925 13:11:38.825098  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 12 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:38.825189  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:38.825377  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:38.826078  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 12.12 11.11
I20260925 13:11:38.927314  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 12.12 11.11
I20260925 13:11:39.128358  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 12.12 11.11
I20260925 13:11:39.229094  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 11 FOLLOWER]: Advancing to term 12
I20260925 13:11:39.229878  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 12 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260925 13:11:39.230173  1110 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260925 13:11:39.430544  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:39.430632  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 12 LEADER]: Emulating election...
I20260925 13:11:39.430657  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 12 LEADER]: Stepping down as leader of term 12
I20260925 13:11:39.430672  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:39.430706  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:39.430814  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 12 FOLLOWER]: Advancing to term 13
I20260925 13:11:39.431615  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 13 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:39.431715  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:39.431896  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:39.432929  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 12.12
I20260925 13:11:39.533902  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 12.12
I20260925 13:11:39.734815  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 12.12
I20260925 13:11:39.911275  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 12 FOLLOWER]: Advancing to term 13
I20260925 13:11:39.912142  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 13 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260925 13:11:39.912477  1110 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 13:11:40.037199  1066 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:40.037338  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 13 FOLLOWER]: Emulating election...
I20260925 13:11:40.037364  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 13 FOLLOWER]: Advancing to term 14
I20260925 13:11:40.038120  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 14 LEADER]: Becoming Leader. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:40.038206  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:40.038384  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:40.039557  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 14.14
I20260925 13:11:40.140779  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 14.14
I20260925 13:11:40.341992  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 13.13 14.14
I20260925 13:11:40.438337  1020 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 14 LEADER]: Rejecting Update request from peer 34e4e0667a4742fa96f84f527c47518f for earlier term 13. Current term is 14. Ops: []
I20260925 13:11:40.438695  1121 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260925 13:11:40.438901  1121 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Peer 41497c801aa9425099b05ace25365b8b log is divergent from this leader: its last log entry 14.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260925 13:11:40.439018  1122 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 13 LEADER]: Stepping down as leader of term 13
I20260925 13:11:40.439061  1122 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:40.439097  1122 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:40.439186  1122 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 13 FOLLOWER]: Advancing to term 14
I20260925 13:11:40.627761   944 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 14 FOLLOWER]: Refusing update from remote peer 41497c801aa9425099b05ace25365b8b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260925 13:11:40.628110  1123 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260925 13:11:40.644500  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:40.644618  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 14 FOLLOWER]: Emulating election...
I20260925 13:11:40.644650  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 14 FOLLOWER]: Advancing to term 15
I20260925 13:11:40.645493  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:40.645586  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:40.645807  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:40.646456  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 15.15 14.14
I20260925 13:11:40.747641  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 15.15 14.14
I20260925 13:11:40.948982  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 15.15 14.14
I20260925 13:11:41.031359   944 raft_consensus.cc:1226] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 LEADER]: Rejecting Update request from peer 41497c801aa9425099b05ace25365b8b for earlier term 14. Current term is 15. Ops: []
I20260925 13:11:41.031689  1123 consensus_queue.cc:1046] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260925 13:11:41.031816  1123 consensus_queue.cc:1225] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [LEADER]: Peer 34e4e0667a4742fa96f84f527c47518f log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260925 13:11:41.031858  1124 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 14 LEADER]: Stepping down as leader of term 14
I20260925 13:11:41.031953  1124 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41497c801aa9425099b05ace25365b8b, State: Running, Role: LEADER
I20260925 13:11:41.032014  1124 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:41.032130  1124 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 14 FOLLOWER]: Advancing to term 15
I20260925 13:11:41.073704  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 15 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260925 13:11:41.074057  1122 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260925 13:11:41.251507  1066 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:41.251586  1066 raft_consensus.cc:412] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 LEADER]: Emulating election...
I20260925 13:11:41.251616  1066 raft_consensus.cc:3032] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 LEADER]: Stepping down as leader of term 15
I20260925 13:11:41.251629  1066 raft_consensus.cc:729] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:41.251700  1066 consensus_queue.cc:260] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:41.251824  1066 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 15 FOLLOWER]: Advancing to term 16
I20260925 13:11:41.254498  1066 raft_consensus.cc:686] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 16 LEADER]: Becoming Leader. State: Replica: 34e4e0667a4742fa96f84f527c47518f, State: Running, Role: LEADER
I20260925 13:11:41.254611  1066 consensus_queue.cc:237] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 } } peers { permanent_uuid: "34e4e0667a4742fa96f84f527c47518f" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 45915 } }
I20260925 13:11:41.254920  1066 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:41.255689  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 16.16 15.15
I20260925 13:11:41.356654  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 16.16 15.15
I20260925 13:11:41.557915  1066 cluster_itest_util.cc:256] Not converged past 0 yet: 16.16 15.15
I20260925 13:11:41.778306  1020 raft_consensus.cc:3037] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 15 FOLLOWER]: Advancing to term 16
I20260925 13:11:41.779023  1020 raft_consensus.cc:1261] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 16 FOLLOWER]: Refusing update from remote peer 34e4e0667a4742fa96f84f527c47518f: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260925 13:11:41.779392  1128 consensus_queue.cc:1035] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41497c801aa9425099b05ace25365b8b" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260925 13:11:41.859680   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:41.863032   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:41.863181   392 tablet_replica.cc:330] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f: stopping tablet replica
I20260925 13:11:41.863252   392 raft_consensus.cc:2229] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 16 LEADER]: Raft consensus shutting down.
I20260925 13:11:41.863317   392 raft_consensus.cc:2258] T 2f21f740158b4b888ed8a92b0c98305d P 34e4e0667a4742fa96f84f527c47518f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:41.875617   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:41.876822   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:41.879680   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:41.879842   392 tablet_replica.cc:330] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b: stopping tablet replica
I20260925 13:11:41.879932   392 raft_consensus.cc:2229] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:41.879987   392 raft_consensus.cc:2258] T 2f21f740158b4b888ed8a92b0c98305d P 41497c801aa9425099b05ace25365b8b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:41.892267   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:41.893534   392 master.cc:537] Master@127.0.98.62:34989 shutting down...
I20260925 13:11:41.896903   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:41.896983   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:41.897006   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 51f2b60e0c9d45b4b8f680cfc8c380df: stopping tablet replica
I20260925 13:11:41.909415   392 master.cc:552] Master@127.0.98.62:34989 shutdown complete.
[       OK ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (10309 ms)
[ RUN      ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260925 13:11:41.914180   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:38385
I20260925 13:11:41.914414   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:41.916563  1137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:41.916733  1140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:41.916615  1138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:41.916615   392 server_base.cc:1034] running on GCE node
I20260925 13:11:41.916965   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:41.917006   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:41.917038   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341901917039 us; error 0 us; skew 500 ppm
I20260925 13:11:41.917781   392 webserver.cc:460] Webserver started at http://127.0.98.62:46607/ using document root <none> and password file <none>
I20260925 13:11:41.917855   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:41.917891   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:41.917927   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:41.918172   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e"
format_stamp: "Formatted at 2026-09-25 13:11:41 on dist-test-slave-2538"
I20260925 13:11:41.919116   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:41.919621  1145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:41.919812   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:41.919868   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e"
format_stamp: "Formatted at 2026-09-25 13:11:41 on dist-test-slave-2538"
I20260925 13:11:41.919921   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:41.948954   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:41.949290   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:41.951785   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:38385
I20260925 13:11:41.953028  1197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:38385 every 8 connection(s)
I20260925 13:11:41.953164  1198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:41.954180  1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e: Bootstrap starting.
I20260925 13:11:41.954417  1198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:41.954936  1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e: No bootstrap required, opened a new log
I20260925 13:11:41.955039  1198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER }
I20260925 13:11:41.955104  1198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:41.955122  1198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2157d3e53ce54176a7d60c6e0dd1aa6e, State: Initialized, Role: FOLLOWER
I20260925 13:11:41.955194  1198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER }
I20260925 13:11:41.955246  1198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:41.955262  1198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:41.955278  1198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:41.955755  1198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER }
I20260925 13:11:41.955807  1198 leader_election.cc:302] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2157d3e53ce54176a7d60c6e0dd1aa6e; no voters: 
I20260925 13:11:41.955892  1198 leader_election.cc:288] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:41.955940  1201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:41.956084  1198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:41.956147  1201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 2157d3e53ce54176a7d60c6e0dd1aa6e, State: Running, Role: LEADER
I20260925 13:11:41.956202  1201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER }
I20260925 13:11:41.956477  1202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER } }
I20260925 13:11:41.956571  1202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:41.956735  1209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:41.956486  1203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2157d3e53ce54176a7d60c6e0dd1aa6e. Latest consensus state: current_term: 1 leader_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2157d3e53ce54176a7d60c6e0dd1aa6e" member_type: VOTER } }
I20260925 13:11:41.956839  1203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:41.956915  1209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:41.957361   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:41.957569  1209 catalog_manager.cc:1347] Generated new cluster ID: 34875355bdb14e8fbf174d6dd290af78
I20260925 13:11:41.957623  1209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:41.966459  1209 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:41.966854  1209 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:41.970749  1209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e: Generated new TSK 0
I20260925 13:11:41.970844  1209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:41.973281   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:41.975250   392 server_base.cc:1034] running on GCE node
W20260925 13:11:41.975291  1222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:41.975371  1220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:41.975449  1219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:41.975669   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:41.975739   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:41.975762   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341901975762 us; error 0 us; skew 500 ppm
I20260925 13:11:41.976593   392 webserver.cc:460] Webserver started at http://127.0.98.1:41401/ using document root <none> and password file <none>
I20260925 13:11:41.976713   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:41.976766   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:41.976818   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:41.977250   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "3a17a5724916497286c89a07b14d0ac0"
format_stamp: "Formatted at 2026-09-25 13:11:41 on dist-test-slave-2538"
I20260925 13:11:41.978392   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:41.979079  1227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:41.979373   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:41.979437   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "3a17a5724916497286c89a07b14d0ac0"
format_stamp: "Formatted at 2026-09-25 13:11:41 on dist-test-slave-2538"
I20260925 13:11:41.979491   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:42.004978   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:42.005290   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:42.005501   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:42.005750   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:42.005801   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:42.005849   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:42.005879   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:42.008893   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:38819
I20260925 13:11:42.008967  1290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:38819 every 8 connection(s)
I20260925 13:11:42.009416   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:42.010963  1295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:42.011162  1298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:42.011250   392 server_base.cc:1034] running on GCE node
W20260925 13:11:42.011149  1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:42.011512   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:42.011571   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:42.011585   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341902011585 us; error 0 us; skew 500 ppm
I20260925 13:11:42.012516   392 webserver.cc:460] Webserver started at http://127.0.98.2:35093/ using document root <none> and password file <none>
I20260925 13:11:42.012621   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:42.012678   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:42.012734   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:42.013195   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "9ae478c53c6b48d78569c684774f7c86"
format_stamp: "Formatted at 2026-09-25 13:11:42 on dist-test-slave-2538"
I20260925 13:11:42.013770  1291 heartbeater.cc:347] Connected to a master server at 127.0.98.62:38385
I20260925 13:11:42.013825  1291 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:42.013908  1291 heartbeater.cc:511] Master 127.0.98.62:38385 requested a full tablet report, sending...
I20260925 13:11:42.014216  1162 ts_manager.cc:194] Registered new tserver with Master: 3a17a5724916497286c89a07b14d0ac0 (127.0.98.1:38819)
I20260925 13:11:42.014639   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:42.014788  1162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59154
I20260925 13:11:42.015390  1303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:42.015514   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:42.015568   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "9ae478c53c6b48d78569c684774f7c86"
format_stamp: "Formatted at 2026-09-25 13:11:42 on dist-test-slave-2538"
I20260925 13:11:42.015623   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:42.029774   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:42.030019   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:42.030216   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:42.030453   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:42.030505   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:42.030551   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:42.030582   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:42.034371   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:41695
I20260925 13:11:42.034446  1366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:41695 every 8 connection(s)
I20260925 13:11:42.038462  1367 heartbeater.cc:347] Connected to a master server at 127.0.98.62:38385
I20260925 13:11:42.038527  1367 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:42.038605  1367 heartbeater.cc:511] Master 127.0.98.62:38385 requested a full tablet report, sending...
I20260925 13:11:42.038924  1162 ts_manager.cc:194] Registered new tserver with Master: 9ae478c53c6b48d78569c684774f7c86 (127.0.98.2:41695)
I20260925 13:11:42.039073   392 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.004456299s
I20260925 13:11:42.039424  1162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59162
I20260925 13:11:42.044777  1162 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59172:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260925 13:11:42.044912  1162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260925 13:11:42.049912  1255 tablet_service.cc:1467] Processing CreateTablet for tablet 81e2064776414f108a2ee1ec703bbfbb (DEFAULT_TABLE table=test-table [id=4b073b99c63146a3b22e99ca97fe6554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:42.050071  1255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e2064776414f108a2ee1ec703bbfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:42.051190  1380 tablet_bootstrap.cc:492] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: Bootstrap starting.
I20260925 13:11:42.051306  1331 tablet_service.cc:1467] Processing CreateTablet for tablet 81e2064776414f108a2ee1ec703bbfbb (DEFAULT_TABLE table=test-table [id=4b073b99c63146a3b22e99ca97fe6554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:42.051415  1331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e2064776414f108a2ee1ec703bbfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:42.051705  1380 tablet_bootstrap.cc:654] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:42.052484  1380 tablet_bootstrap.cc:492] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: No bootstrap required, opened a new log
I20260925 13:11:42.052526  1380 ts_tablet_manager.cc:1397] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:42.052731  1382 tablet_bootstrap.cc:492] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: Bootstrap starting.
I20260925 13:11:42.052731  1380 raft_consensus.cc:348] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.052845  1380 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Initialized, Role: FOLLOWER
I20260925 13:11:42.052891  1380 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.053157  1380 ts_tablet_manager.cc:1428] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:42.053217  1382 tablet_bootstrap.cc:654] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:42.053251  1291 heartbeater.cc:503] Master 127.0.98.62:38385 was elected leader, sending a full tablet report...
I20260925 13:11:42.053937  1382 tablet_bootstrap.cc:492] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: No bootstrap required, opened a new log
I20260925 13:11:42.054001  1382 ts_tablet_manager.cc:1397] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:42.054162  1382 raft_consensus.cc:348] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.054260  1382 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Initialized, Role: FOLLOWER
I20260925 13:11:42.054293  1382 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.054453  1382 ts_tablet_manager.cc:1428] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:42.054486  1367 heartbeater.cc:503] Master 127.0.98.62:38385 was elected leader, sending a full tablet report...
I20260925 13:11:42.059638   392 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:42.059715   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 0 FOLLOWER]: Emulating election...
I20260925 13:11:42.059732   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:42.060272   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:42.060321   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.060587   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:42.066859   392 cluster_itest_util.cc:256] Not converged past 1 yet: 0.0 1.1
I20260925 13:11:42.168071   392 cluster_itest_util.cc:256] Not converged past 1 yet: 0.0 1.1
I20260925 13:11:42.369513   392 cluster_itest_util.cc:256] Not converged past 1 yet: 0.0 1.1
I20260925 13:11:42.464389  1265 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:42.465517  1385 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:42.670898   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:42.670965   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 1 FOLLOWER]: Emulating election...
I20260925 13:11:42.670986   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:42.671644   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:42.671738   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:42.671969   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:42.672839   392 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260925 13:11:42.774308   392 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260925 13:11:42.975508   392 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260925 13:11:43.034714  1265 raft_consensus.cc:1226] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 LEADER]: Rejecting Update request from peer 9ae478c53c6b48d78569c684774f7c86 for earlier term 1. Current term is 2. Ops: []
I20260925 13:11:43.035099  1392 consensus_queue.cc:1046] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:43.035256  1392 consensus_queue.cc:1225] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Peer 3a17a5724916497286c89a07b14d0ac0 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260925 13:11:43.035478  1393 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:43.035533  1393 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:43.035567  1393 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:43.035679  1393 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:43.277119   392 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260925 13:11:43.296111  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 2 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 13:11:43.296434  1394 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:43.678501   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:43.678568   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 LEADER]: Emulating election...
I20260925 13:11:43.678587   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:43.678599   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:43.678622   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:43.678733   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:43.679520   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 3 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:43.679590   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:43.679772   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:43.680378   392 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260925 13:11:43.781747   392 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260925 13:11:43.983191   392 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260925 13:11:44.269327  1341 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:44.270256  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 3 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260925 13:11:44.270568  1400 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:44.284611   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:44.284682   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 3 LEADER]: Emulating election...
I20260925 13:11:44.284701   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 3 LEADER]: Stepping down as leader of term 3
I20260925 13:11:44.284713   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:44.284736   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:44.284802   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:44.285521   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 4 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:44.285574   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:44.285737   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:44.286484   392 cluster_itest_util.cc:256] Not converged past 4 yet: 4.4 3.3
I20260925 13:11:44.387691   392 cluster_itest_util.cc:256] Not converged past 4 yet: 4.4 3.3
I20260925 13:11:44.589105   392 cluster_itest_util.cc:256] Not converged past 4 yet: 4.4 3.3
I20260925 13:11:44.890465   392 cluster_itest_util.cc:256] Not converged past 4 yet: 4.4 3.3
I20260925 13:11:44.902484  1341 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:44.903517  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 4 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260925 13:11:44.903817  1405 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:45.292001   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:45.292076   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 4 LEADER]: Emulating election...
I20260925 13:11:45.292095   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 4 LEADER]: Stepping down as leader of term 4
I20260925 13:11:45.292107   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:45.292140   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:45.292233   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:45.292876   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 5 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:45.292930   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:45.293102   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:45.293682   392 cluster_itest_util.cc:256] Not converged past 5 yet: 5.5 4.4
I20260925 13:11:45.395183   392 cluster_itest_util.cc:256] Not converged past 5 yet: 5.5 4.4
I20260925 13:11:45.596800   392 cluster_itest_util.cc:256] Not converged past 5 yet: 5.5 4.4
I20260925 13:11:45.750448  1341 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:45.751554  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 5 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260925 13:11:45.751927  1407 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:45.898582   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:45.898690   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 5 LEADER]: Emulating election...
I20260925 13:11:45.898720   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 5 LEADER]: Stepping down as leader of term 5
I20260925 13:11:45.898739   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:45.898793   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:45.898916   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:45.899582   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 6 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:45.899667   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:45.899923   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:45.900715   392 cluster_itest_util.cc:256] Not converged past 6 yet: 6.6 5.5
I20260925 13:11:46.002153   392 cluster_itest_util.cc:256] Not converged past 6 yet: 6.6 5.5
I20260925 13:11:46.203681   392 cluster_itest_util.cc:256] Not converged past 6 yet: 6.6 5.5
I20260925 13:11:46.489472  1341 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:46.492539  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 6 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260925 13:11:46.492921  1414 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:46.504983   392 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:46.505072   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 6 FOLLOWER]: Emulating election...
I20260925 13:11:46.505116   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:46.505848   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 LEADER]: Becoming Leader. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:46.505935   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:46.506106   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:46.506942   392 cluster_itest_util.cc:256] Not converged past 7 yet: 6.6 7.7
I20260925 13:11:46.608270   392 cluster_itest_util.cc:256] Not converged past 7 yet: 6.6 7.7
I20260925 13:11:46.809870   392 cluster_itest_util.cc:256] Not converged past 7 yet: 6.6 7.7
I20260925 13:11:47.036820  1341 raft_consensus.cc:1226] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 LEADER]: Rejecting Update request from peer 3a17a5724916497286c89a07b14d0ac0 for earlier term 6. Current term is 7. Ops: []
I20260925 13:11:47.037274  1419 consensus_queue.cc:1046] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:47.037447  1419 consensus_queue.cc:1225] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Peer 9ae478c53c6b48d78569c684774f7c86 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260925 13:11:47.037568  1420 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 6 LEADER]: Stepping down as leader of term 6
I20260925 13:11:47.037621  1420 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:47.037662  1420 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:47.037774  1420 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:47.105798  1265 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 7 FOLLOWER]: Refusing update from remote peer 9ae478c53c6b48d78569c684774f7c86: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260925 13:11:47.106266  1421 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:47.111200   392 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260925 13:11:47.111258   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 LEADER]: Emulating election...
I20260925 13:11:47.111275   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 LEADER]: Stepping down as leader of term 7
I20260925 13:11:47.111289   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:47.111310   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:47.111438   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:47.112196   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 8 LEADER]: Becoming Leader. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:47.112246   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:47.112414   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:47.113019   392 cluster_itest_util.cc:256] Not converged past 8 yet: 7.7 8.8
I20260925 13:11:47.214565   392 cluster_itest_util.cc:256] Not converged past 8 yet: 7.7 8.8
I20260925 13:11:47.416030   392 cluster_itest_util.cc:256] Not converged past 8 yet: 7.7 8.8
I20260925 13:11:47.717350   392 cluster_itest_util.cc:256] Not converged past 8 yet: 7.7 8.8
I20260925 13:11:47.724900  1265 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 7 FOLLOWER]: Advancing to term 8
I20260925 13:11:47.725795  1265 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 8 FOLLOWER]: Refusing update from remote peer 9ae478c53c6b48d78569c684774f7c86: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260925 13:11:47.726071  1425 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:11:48.118594   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:48.118669   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 8 FOLLOWER]: Emulating election...
I20260925 13:11:48.118690   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:48.119438   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:48.119532   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:48.119742   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:48.120718   392 cluster_itest_util.cc:256] Not converged past 9 yet: 9.9 8.8
I20260925 13:11:48.221804   392 cluster_itest_util.cc:256] Not converged past 9 yet: 9.9 8.8
I20260925 13:11:48.249811  1265 raft_consensus.cc:1226] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 LEADER]: Rejecting Update request from peer 9ae478c53c6b48d78569c684774f7c86 for earlier term 8. Current term is 9. Ops: []
I20260925 13:11:48.250164  1430 consensus_queue.cc:1046] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:11:48.250355  1430 consensus_queue.cc:1225] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [LEADER]: Peer 3a17a5724916497286c89a07b14d0ac0 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260925 13:11:48.250432  1431 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 8 LEADER]: Stepping down as leader of term 8
I20260925 13:11:48.250524  1431 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9ae478c53c6b48d78569c684774f7c86, State: Running, Role: LEADER
I20260925 13:11:48.250643  1431 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:48.250746  1431 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 8 FOLLOWER]: Advancing to term 9
I20260925 13:11:48.422921   392 cluster_itest_util.cc:256] Not converged past 9 yet: 9.9 8.8
I20260925 13:11:48.591295  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 9 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260925 13:11:48.591707  1426 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:11:48.724404   392 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260925 13:11:48.724558   392 raft_consensus.cc:412] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 LEADER]: Emulating election...
I20260925 13:11:48.724582   392 raft_consensus.cc:3032] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 LEADER]: Stepping down as leader of term 9
I20260925 13:11:48.724596   392 raft_consensus.cc:729] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:48.724695   392 consensus_queue.cc:260] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:48.724813   392 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:48.725606   392 raft_consensus.cc:686] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 10 LEADER]: Becoming Leader. State: Replica: 3a17a5724916497286c89a07b14d0ac0, State: Running, Role: LEADER
I20260925 13:11:48.725694   392 consensus_queue.cc:237] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 } } peers { permanent_uuid: "3a17a5724916497286c89a07b14d0ac0" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 38819 } }
I20260925 13:11:48.725860   392 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260925 13:11:48.726655   392 cluster_itest_util.cc:256] Not converged past 10 yet: 10.10 9.9
I20260925 13:11:48.827776   392 cluster_itest_util.cc:256] Not converged past 10 yet: 10.10 9.9
I20260925 13:11:49.029064   392 cluster_itest_util.cc:256] Not converged past 10 yet: 10.10 9.9
I20260925 13:11:49.330175   392 cluster_itest_util.cc:256] Not converged past 10 yet: 10.10 9.9
I20260925 13:11:49.340292  1341 raft_consensus.cc:3037] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 9 FOLLOWER]: Advancing to term 10
I20260925 13:11:49.341212  1341 raft_consensus.cc:1261] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 10 FOLLOWER]: Refusing update from remote peer 3a17a5724916497286c89a07b14d0ac0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260925 13:11:49.341492  1435 consensus_queue.cc:1035] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae478c53c6b48d78569c684774f7c86" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:11:49.731623   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:49.734297   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:49.734525   392 tablet_replica.cc:330] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0: stopping tablet replica
I20260925 13:11:49.734584   392 raft_consensus.cc:2229] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 10 LEADER]: Raft consensus shutting down.
I20260925 13:11:49.734644   392 raft_consensus.cc:2258] T 81e2064776414f108a2ee1ec703bbfbb P 3a17a5724916497286c89a07b14d0ac0 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:49.736508   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:49.737367   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:49.739948   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:49.740070   392 tablet_replica.cc:330] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86: stopping tablet replica
I20260925 13:11:49.740144   392 raft_consensus.cc:2229] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:49.740192   392 raft_consensus.cc:2258] T 81e2064776414f108a2ee1ec703bbfbb P 9ae478c53c6b48d78569c684774f7c86 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:49.752130   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:49.753037   392 master.cc:537] Master@127.0.98.62:38385 shutting down...
I20260925 13:11:49.756181   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:49.756265   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:49.756285   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2157d3e53ce54176a7d60c6e0dd1aa6e: stopping tablet replica
I20260925 13:11:49.767648   392 master.cc:552] Master@127.0.98.62:38385 shutdown complete.
[       OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (7857 ms)
[----------] 3 tests from LeadershipChangeReportingTest (32995 ms total)

[----------] 4 tests from TsTabletManagerITest
[ RUN      ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260925 13:11:49.771782   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:34843
I20260925 13:11:49.771950   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:49.773994   392 server_base.cc:1034] running on GCE node
W20260925 13:11:49.774127  1447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.774163  1445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.774225  1444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:49.774382   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:49.774425   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:49.774437   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341909774437 us; error 0 us; skew 500 ppm
I20260925 13:11:49.781948   392 webserver.cc:460] Webserver started at http://127.0.98.62:44901/ using document root <none> and password file <none>
I20260925 13:11:49.782094   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:49.782152   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:49.782202   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:49.782492   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "b1bc71507bbd4c46a4827778b41b9ddc"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.783541   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:49.784137  1452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.784293   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:49.784348   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "b1bc71507bbd4c46a4827778b41b9ddc"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.784478   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:49.794282   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:49.794474   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:49.797020   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:34843
I20260925 13:11:49.798391  1505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:49.798440  1504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:34843 every 8 connection(s)
I20260925 13:11:49.799463  1505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc: Bootstrap starting.
I20260925 13:11:49.799770  1505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:49.800341  1505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc: No bootstrap required, opened a new log
I20260925 13:11:49.800578  1505 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER }
I20260925 13:11:49.800680  1505 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:49.800714  1505 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bc71507bbd4c46a4827778b41b9ddc, State: Initialized, Role: FOLLOWER
I20260925 13:11:49.800770  1505 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER }
I20260925 13:11:49.800850  1505 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:49.800907  1505 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:49.800940  1505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:49.801519  1505 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER }
I20260925 13:11:49.801605  1505 leader_election.cc:302] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1bc71507bbd4c46a4827778b41b9ddc; no voters: 
I20260925 13:11:49.801718  1505 leader_election.cc:288] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:49.801769  1508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:49.801882  1505 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:49.801932  1508 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 1 LEADER]: Becoming Leader. State: Replica: b1bc71507bbd4c46a4827778b41b9ddc, State: Running, Role: LEADER
I20260925 13:11:49.801987  1508 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER }
I20260925 13:11:49.802289  1509 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER } }
I20260925 13:11:49.802428  1509 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:49.802300  1510 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1bc71507bbd4c46a4827778b41b9ddc. Latest consensus state: current_term: 1 leader_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bc71507bbd4c46a4827778b41b9ddc" member_type: VOTER } }
I20260925 13:11:49.802583  1510 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:49.802680  1518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:49.802847  1518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:49.803123   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:49.803485  1518 catalog_manager.cc:1347] Generated new cluster ID: fdd93bee28704b2e89a1d2372e82352a
I20260925 13:11:49.803532  1518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:49.817946  1518 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:49.818400  1518 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:49.824251  1518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc: Generated new TSK 0
I20260925 13:11:49.824342  1518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:49.834925   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:49.836627  1527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.836580  1529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.836606  1526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:49.836885   392 server_base.cc:1034] running on GCE node
I20260925 13:11:49.837003   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:49.837040   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:49.837052   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341909837053 us; error 0 us; skew 500 ppm
I20260925 13:11:49.837889   392 webserver.cc:460] Webserver started at http://127.0.98.1:42063/ using document root <none> and password file <none>
I20260925 13:11:49.837961   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:49.837994   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:49.838029   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:49.838400   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "c36e24b9ac1e4e43b4b845105b872771"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.839314   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:49.839823  1534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.839958   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.840011   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "c36e24b9ac1e4e43b4b845105b872771"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.840065   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:49.865518   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:49.865710   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:49.865839   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:49.866063   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:49.866093   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.866115   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:49.866163   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.868968   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:34437
I20260925 13:11:49.869002  1597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:34437 every 8 connection(s)
I20260925 13:11:49.869380   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:49.870882  1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.870971  1605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.870971  1602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:49.871245   392 server_base.cc:1034] running on GCE node
I20260925 13:11:49.871376   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:49.871411   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:49.871428   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341909871428 us; error 0 us; skew 500 ppm
I20260925 13:11:49.872215   392 webserver.cc:460] Webserver started at http://127.0.98.2:39851/ using document root <none> and password file <none>
I20260925 13:11:49.872318   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:49.872364   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:49.872413   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:49.872695   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "0422dd182bd849a785bfdfc3199a8961"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.873693  1598 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34843
I20260925 13:11:49.873756  1598 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:49.873834  1598 heartbeater.cc:511] Master 127.0.98.62:34843 requested a full tablet report, sending...
I20260925 13:11:49.873991   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:49.874090  1469 ts_manager.cc:194] Registered new tserver with Master: c36e24b9ac1e4e43b4b845105b872771 (127.0.98.1:34437)
I20260925 13:11:49.874572  1610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.874598  1469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46446
I20260925 13:11:49.874694   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:49.874742   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "0422dd182bd849a785bfdfc3199a8961"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.874791   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:49.894447   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:49.894719   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:49.894910   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:49.895123   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:49.895169   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.895202   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:49.895229   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.898578   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:37059
I20260925 13:11:49.898605  1673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:37059 every 8 connection(s)
I20260925 13:11:49.898975   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:49.900633  1680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:49.900701  1677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:49.900858   392 server_base.cc:1034] running on GCE node
W20260925 13:11:49.901047  1678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:49.901258   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:49.901298   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:49.901325   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341909901325 us; error 0 us; skew 500 ppm
I20260925 13:11:49.902086   392 webserver.cc:460] Webserver started at http://127.0.98.3:39287/ using document root <none> and password file <none>
I20260925 13:11:49.902175   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:49.902221   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:49.902272   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:49.902611   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-2-root/instance:
uuid: "0916b27904434d50a3e577692bf1d702"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.903200  1674 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34843
I20260925 13:11:49.903277  1674 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:49.903359  1674 heartbeater.cc:511] Master 127.0.98.62:34843 requested a full tablet report, sending...
I20260925 13:11:49.903642  1469 ts_manager.cc:194] Registered new tserver with Master: 0422dd182bd849a785bfdfc3199a8961 (127.0.98.2:37059)
I20260925 13:11:49.903836   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20260925 13:11:49.904095  1469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46450
I20260925 13:11:49.904397  1685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.904529   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:49.904577   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-2-root
uuid: "0916b27904434d50a3e577692bf1d702"
format_stamp: "Formatted at 2026-09-25 13:11:49 on dist-test-slave-2538"
I20260925 13:11:49.904627   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790341876762790-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:49.916630   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:49.916888   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:49.917114   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:49.917322   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:49.917369   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.917410   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:49.917435   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:49.920707   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:46321
I20260925 13:11:49.920785  1748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:46321 every 8 connection(s)
I20260925 13:11:49.924870  1749 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34843
I20260925 13:11:49.924932  1749 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:49.925052  1749 heartbeater.cc:511] Master 127.0.98.62:34843 requested a full tablet report, sending...
I20260925 13:11:49.925323  1469 ts_manager.cc:194] Registered new tserver with Master: 0916b27904434d50a3e577692bf1d702 (127.0.98.3:46321)
I20260925 13:11:49.925488   392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004532236s
I20260925 13:11:49.925822  1469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46464
I20260925 13:11:49.931334  1469 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46470:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
  }
}
W20260925 13:11:49.931457  1469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:49.936939  1638 tablet_service.cc:1467] Processing CreateTablet for tablet 335c14c5202b4fcd8e8154ac402faec6 (DEFAULT_TABLE table=test-table [id=538373d451d147eb9ce72b6eb7eec7a9]), partition=
I20260925 13:11:49.937126  1638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335c14c5202b4fcd8e8154ac402faec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:49.937181  1562 tablet_service.cc:1467] Processing CreateTablet for tablet 335c14c5202b4fcd8e8154ac402faec6 (DEFAULT_TABLE table=test-table [id=538373d451d147eb9ce72b6eb7eec7a9]), partition=
I20260925 13:11:49.937255  1562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335c14c5202b4fcd8e8154ac402faec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:49.937565  1713 tablet_service.cc:1467] Processing CreateTablet for tablet 335c14c5202b4fcd8e8154ac402faec6 (DEFAULT_TABLE table=test-table [id=538373d451d147eb9ce72b6eb7eec7a9]), partition=
I20260925 13:11:49.937669  1713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335c14c5202b4fcd8e8154ac402faec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:49.938925  1765 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: Bootstrap starting.
I20260925 13:11:49.939035  1766 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: Bootstrap starting.
I20260925 13:11:49.939445  1765 tablet_bootstrap.cc:654] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:49.939445  1766 tablet_bootstrap.cc:654] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:49.939919  1764 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: Bootstrap starting.
I20260925 13:11:49.940156  1765 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: No bootstrap required, opened a new log
I20260925 13:11:49.940212  1765 ts_tablet_manager.cc:1397] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:49.940294  1766 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: No bootstrap required, opened a new log
I20260925 13:11:49.940349  1766 ts_tablet_manager.cc:1397] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:49.940371  1764 tablet_bootstrap.cc:654] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:49.940421  1765 raft_consensus.cc:348] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.940521  1766 raft_consensus.cc:348] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.940616  1766 raft_consensus.cc:374] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:49.940651  1766 raft_consensus.cc:729] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0916b27904434d50a3e577692bf1d702, State: Initialized, Role: FOLLOWER
I20260925 13:11:49.940711  1766 consensus_queue.cc:260] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.940860  1766 ts_tablet_manager.cc:1428] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:49.940526  1765 raft_consensus.cc:374] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:49.940982  1749 heartbeater.cc:503] Master 127.0.98.62:34843 was elected leader, sending a full tablet report...
I20260925 13:11:49.941012  1764 tablet_bootstrap.cc:492] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: No bootstrap required, opened a new log
I20260925 13:11:49.941025  1765 raft_consensus.cc:729] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e24b9ac1e4e43b4b845105b872771, State: Initialized, Role: FOLLOWER
I20260925 13:11:49.941049  1764 ts_tablet_manager.cc:1397] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:49.941147  1765 consensus_queue.cc:260] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.941255  1764 raft_consensus.cc:348] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.941340  1765 ts_tablet_manager.cc:1428] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:49.941426  1598 heartbeater.cc:503] Master 127.0.98.62:34843 was elected leader, sending a full tablet report...
I20260925 13:11:49.941330  1764 raft_consensus.cc:374] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:49.941471  1764 raft_consensus.cc:729] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0422dd182bd849a785bfdfc3199a8961, State: Initialized, Role: FOLLOWER
I20260925 13:11:49.941545  1764 consensus_queue.cc:260] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:49.941726  1764 ts_tablet_manager.cc:1428] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:49.941831  1674 heartbeater.cc:503] Master 127.0.98.62:34843 was elected leader, sending a full tablet report...
W20260925 13:11:50.119416  1599 tablet.cc:2369] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:50.183509  1770 raft_consensus.cc:482] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:50.183616  1770 raft_consensus.cc:504] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:50.183928  1770 leader_election.cc:288] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0422dd182bd849a785bfdfc3199a8961 (127.0.98.2:37059), c36e24b9ac1e4e43b4b845105b872771 (127.0.98.1:34437)
I20260925 13:11:50.187276  1648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335c14c5202b4fcd8e8154ac402faec6" candidate_uuid: "0916b27904434d50a3e577692bf1d702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0422dd182bd849a785bfdfc3199a8961" is_pre_election: true
I20260925 13:11:50.187279  1572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335c14c5202b4fcd8e8154ac402faec6" candidate_uuid: "0916b27904434d50a3e577692bf1d702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e24b9ac1e4e43b4b845105b872771" is_pre_election: true
I20260925 13:11:50.187426  1648 raft_consensus.cc:2454] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0916b27904434d50a3e577692bf1d702 in term 0.
I20260925 13:11:50.187429  1572 raft_consensus.cc:2454] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0916b27904434d50a3e577692bf1d702 in term 0.
I20260925 13:11:50.187623  1687 leader_election.cc:302] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0422dd182bd849a785bfdfc3199a8961, 0916b27904434d50a3e577692bf1d702; no voters: 
I20260925 13:11:50.187739  1770 raft_consensus.cc:2781] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:50.187806  1770 raft_consensus.cc:482] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:50.187839  1770 raft_consensus.cc:3037] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:50.188503  1770 raft_consensus.cc:504] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:50.188644  1770 leader_election.cc:288] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [CANDIDATE]: Term 1 election: Requested vote from peers 0422dd182bd849a785bfdfc3199a8961 (127.0.98.2:37059), c36e24b9ac1e4e43b4b845105b872771 (127.0.98.1:34437)
I20260925 13:11:50.188834  1648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335c14c5202b4fcd8e8154ac402faec6" candidate_uuid: "0916b27904434d50a3e577692bf1d702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0422dd182bd849a785bfdfc3199a8961"
I20260925 13:11:50.188926  1648 raft_consensus.cc:3037] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:50.188934  1572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335c14c5202b4fcd8e8154ac402faec6" candidate_uuid: "0916b27904434d50a3e577692bf1d702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e24b9ac1e4e43b4b845105b872771"
I20260925 13:11:50.188999  1572 raft_consensus.cc:3037] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:50.189677  1572 raft_consensus.cc:2454] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0916b27904434d50a3e577692bf1d702 in term 1.
I20260925 13:11:50.189677  1648 raft_consensus.cc:2454] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0916b27904434d50a3e577692bf1d702 in term 1.
I20260925 13:11:50.189845  1687 leader_election.cc:302] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0422dd182bd849a785bfdfc3199a8961, 0916b27904434d50a3e577692bf1d702; no voters: 
I20260925 13:11:50.189913  1770 raft_consensus.cc:2781] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:50.190053  1770 raft_consensus.cc:686] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 1 LEADER]: Becoming Leader. State: Replica: 0916b27904434d50a3e577692bf1d702, State: Running, Role: LEADER
I20260925 13:11:50.190097  1770 consensus_queue.cc:237] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } }
I20260925 13:11:50.190687  1469 catalog_manager.cc:5495] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0916b27904434d50a3e577692bf1d702 (127.0.98.3). New cstate: current_term: 1 leader_uuid: "0916b27904434d50a3e577692bf1d702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0916b27904434d50a3e577692bf1d702" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46321 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:50.731850  1777 consensus_queue.cc:1035] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e24b9ac1e4e43b4b845105b872771" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 34437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:50.733645  1777 consensus_queue.cc:1035] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0422dd182bd849a785bfdfc3199a8961" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 37059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:52.247144   392 tablet_replica.cc:330] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771: stopping tablet replica
I20260925 13:11:52.247242   392 raft_consensus.cc:2229] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:52.247277   392 raft_consensus.cc:2258] T 335c14c5202b4fcd8e8154ac402faec6 P c36e24b9ac1e4e43b4b845105b872771 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:52.560770  1687 consensus_queue.cc:579] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [LEADER]: The tablet replica hosted on peer c36e24b9ac1e4e43b4b845105b872771 has failed
W20260925 13:11:52.560957  1687 consensus_peers.cc:487] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 -> Peer c36e24b9ac1e4e43b4b845105b872771 (127.0.98.1:34437): Couldn't send request to peer c36e24b9ac1e4e43b4b845105b872771. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:52.760636   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:52.763880   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:52.776095   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:52.777263   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:52.780277   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:52.780459   392 tablet_replica.cc:330] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961: stopping tablet replica
I20260925 13:11:52.780520   392 raft_consensus.cc:2229] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:52.780586   392 raft_consensus.cc:2258] T 335c14c5202b4fcd8e8154ac402faec6 P 0422dd182bd849a785bfdfc3199a8961 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:52.792338   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:52.793253   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:11:52.795647   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:52.795799   392 tablet_replica.cc:330] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702: stopping tablet replica
I20260925 13:11:52.795843   392 raft_consensus.cc:2229] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:52.795946   392 raft_consensus.cc:2258] T 335c14c5202b4fcd8e8154ac402faec6 P 0916b27904434d50a3e577692bf1d702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:52.807559   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:11:52.808403   392 master.cc:537] Master@127.0.98.62:34843 shutting down...
I20260925 13:11:52.810655   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:52.810715   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:52.810734   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P b1bc71507bbd4c46a4827778b41b9ddc: stopping tablet replica
I20260925 13:11:52.822005   392 master.cc:552] Master@127.0.98.62:34843 shutdown complete.
[       OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (3055 ms)
[ RUN      ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260925 13:11:52.826952   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:40029
I20260925 13:11:52.827137   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:52.828953   392 server_base.cc:1034] running on GCE node
W20260925 13:11:52.828960  1796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.828886  1795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.828960  1798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.829474   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:52.829550   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:52.829578   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341912829578 us; error 0 us; skew 500 ppm
I20260925 13:11:52.830226   392 webserver.cc:460] Webserver started at http://127.0.98.62:46841/ using document root <none> and password file <none>
I20260925 13:11:52.830343   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:52.830391   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:52.830438   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:52.830740   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "a369c688ef404750b6056adb802edd3d"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.831707   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:52.832180  1803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.832307   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.832365   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "a369c688ef404750b6056adb802edd3d"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.832437   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:52.841377   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:52.841576   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:52.844341   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:40029
I20260925 13:11:52.845901  1855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:40029 every 8 connection(s)
I20260925 13:11:52.846045  1856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:52.847091  1856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d: Bootstrap starting.
I20260925 13:11:52.847391  1856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:52.847913  1856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d: No bootstrap required, opened a new log
I20260925 13:11:52.848042  1856 raft_consensus.cc:348] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER }
I20260925 13:11:52.848104  1856 raft_consensus.cc:374] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:52.848129  1856 raft_consensus.cc:729] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a369c688ef404750b6056adb802edd3d, State: Initialized, Role: FOLLOWER
I20260925 13:11:52.848198  1856 consensus_queue.cc:260] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER }
I20260925 13:11:52.848241  1856 raft_consensus.cc:388] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:52.848260  1856 raft_consensus.cc:482] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:52.848284  1856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:52.848809  1856 raft_consensus.cc:504] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER }
I20260925 13:11:52.848863  1856 leader_election.cc:302] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a369c688ef404750b6056adb802edd3d; no voters: 
I20260925 13:11:52.848976  1856 leader_election.cc:288] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:52.849030  1859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:52.849155  1856 sys_catalog.cc:564] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:52.849329  1859 raft_consensus.cc:686] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 1 LEADER]: Becoming Leader. State: Replica: a369c688ef404750b6056adb802edd3d, State: Running, Role: LEADER
I20260925 13:11:52.849418  1859 consensus_queue.cc:237] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER }
I20260925 13:11:52.849668  1861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a369c688ef404750b6056adb802edd3d. Latest consensus state: current_term: 1 leader_uuid: "a369c688ef404750b6056adb802edd3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER } }
I20260925 13:11:52.849663  1860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a369c688ef404750b6056adb802edd3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a369c688ef404750b6056adb802edd3d" member_type: VOTER } }
I20260925 13:11:52.849761  1861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:52.849785  1860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:52.850203  1867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:52.850332  1867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:52.850421   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:52.851051  1867 catalog_manager.cc:1347] Generated new cluster ID: 8a71c80316e74edfb08047ecd70ee404
I20260925 13:11:52.851107  1867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:52.863910  1867 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:52.864431  1867 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:52.873580  1867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d: Generated new TSK 0
I20260925 13:11:52.873675  1867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:52.882287   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:52.883903  1880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.884060  1878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.884087  1877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.884268   392 server_base.cc:1034] running on GCE node
I20260925 13:11:52.884378   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:52.884415   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:52.884456   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341912884456 us; error 0 us; skew 500 ppm
I20260925 13:11:52.885250   392 webserver.cc:460] Webserver started at http://127.0.98.1:40423/ using document root <none> and password file <none>
I20260925 13:11:52.885329   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:52.885397   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:52.885468   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:52.885774   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "52cddb7691474d44b7cbd6b6374df284"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.886696   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:52.887539  1885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.887733   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:52.887802   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "52cddb7691474d44b7cbd6b6374df284"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.887877   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:52.900422   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:52.900646   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:52.900820   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:52.901042   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:52.901108   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.901150   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:52.901180   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.904443   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:35335
I20260925 13:11:52.904491  1948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:35335 every 8 connection(s)
I20260925 13:11:52.910329  1949 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40029
I20260925 13:11:52.910399  1949 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:52.910482  1949 heartbeater.cc:511] Master 127.0.98.62:40029 requested a full tablet report, sending...
I20260925 13:11:52.910813  1820 ts_manager.cc:194] Registered new tserver with Master: 52cddb7691474d44b7cbd6b6374df284 (127.0.98.1:35335)
I20260925 13:11:52.911265   392 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006616967s
I20260925 13:11:52.911543  1820 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57328
I20260925 13:11:52.916544   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:52.918871   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:52.930469   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:52.931545   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:52.931782   392 tablet_server.cc:93] deduplicated master addresses: 127.0.98.62:40029
W20260925 13:11:52.933126  1959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.933267   392 server_base.cc:1034] running on GCE node
W20260925 13:11:52.933382  1960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.933400  1962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.933562   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:52.933593   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:52.933605   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341912933606 us; error 0 us; skew 500 ppm
I20260925 13:11:52.934427   392 webserver.cc:460] Webserver started at http://127.0.98.1:40423/ using document root <none> and password file <none>
I20260925 13:11:52.934525   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:52.934566   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:52.935258   392 fs_manager.cc:709] Time spent opening directory manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:52.935712  1967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.935853   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.935906   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "52cddb7691474d44b7cbd6b6374df284"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.935972   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:52.950835   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:52.951046   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:52.951221   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:52.951469   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:52.951517   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.951552   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:52.951579   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.954638   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:35335
I20260925 13:11:52.954672  2030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:35335 every 8 connection(s)
I20260925 13:11:52.955340   392 tablet_server.cc:178] TabletServer@127.0.98.1:35335 shutting down...
I20260925 13:11:52.959378  2031 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40029
I20260925 13:11:52.959444  2031 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:52.959522  2031 heartbeater.cc:511] Master 127.0.98.62:40029 requested a full tablet report, sending...
I20260925 13:11:52.959782  1820 ts_manager.cc:194] Re-registered known tserver with Master: 52cddb7691474d44b7cbd6b6374df284 (127.0.98.1:35335)
I20260925 13:11:52.960139  1820 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57342
I20260925 13:11:52.960606   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:52.972281   392 tablet_server.cc:194] TabletServer@127.0.98.1:35335 shutdown complete.
I20260925 13:11:52.973244   392 master.cc:537] Master@127.0.98.62:40029 shutting down...
I20260925 13:11:52.975620   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:52.975680   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:52.975739   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P a369c688ef404750b6056adb802edd3d: stopping tablet replica
I20260925 13:11:52.977367   392 master.cc:552] Master@127.0.98.62:40029 shutdown complete.
[       OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (153 ms)
[ RUN      ] TsTabletManagerITest.TestTableStats
I20260925 13:11:52.980479   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:40169,127.0.98.61:46353,127.0.98.60:34327
I20260925 13:11:52.980615   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:52.982008  2038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.982163  2039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.982183   392 server_base.cc:1034] running on GCE node
W20260925 13:11:52.982214  2041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.982412   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:52.982455   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:52.982481   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341912982480 us; error 0 us; skew 500 ppm
I20260925 13:11:52.983150   392 webserver.cc:460] Webserver started at http://127.0.98.62:39179/ using document root <none> and password file <none>
I20260925 13:11:52.983265   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:52.983336   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:52.983398   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:52.983680   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "6b61637d5b2d427980a42ef58d9f7221"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.984561   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:52.985031  2046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.985195   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.985255   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "6b61637d5b2d427980a42ef58d9f7221"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.985306   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:52.989828   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:52.990023   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:52.992766   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:40169
I20260925 13:11:52.993463   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:52.994290  2098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:40169 every 8 connection(s)
I20260925 13:11:52.994336  2099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:52.995234  2099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:11:52.995321  2101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:52.995381  2104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.995396   392 server_base.cc:1034] running on GCE node
W20260925 13:11:52.995532  2102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:52.995795   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:52.995831   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:52.995864   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341912995863 us; error 0 us; skew 500 ppm
I20260925 13:11:52.996520   392 webserver.cc:460] Webserver started at http://127.0.98.61:42569/ using document root <none> and password file <none>
I20260925 13:11:52.996604   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:52.996637   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:52.996686   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:52.996958   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root/instance:
uuid: "fd8c0be10d6644cd822c356e727e40fd"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.998018   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:52.998615  2111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:52.998993   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:52.999028   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
uuid: "fd8c0be10d6644cd822c356e727e40fd"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:52.999094   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:53.000044  2099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:11:53.000411  2050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
W20260925 13:11:53.000826  2099 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.61:46353: Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
I20260925 13:11:53.007252   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:53.007431   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:53.010187   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:46353
I20260925 13:11:53.010219  2164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:46353 every 8 connection(s)
I20260925 13:11:53.010511  2165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.010593   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:53.011310  2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:11:53.011931  2170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.011986  2167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:53.012095   392 server_base.cc:1034] running on GCE node
W20260925 13:11:53.012176  2168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:53.012301   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:53.012338   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:53.012355   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341913012356 us; error 0 us; skew 500 ppm
I20260925 13:11:53.013032   392 webserver.cc:460] Webserver started at http://127.0.98.60:34775/ using document root <none> and password file <none>
I20260925 13:11:53.013157   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:53.013206   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:53.013258   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:53.013497   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root/instance:
uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.014410   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:53.014927  2176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.015080   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.015136   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.015187   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:53.015236  2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:53.018484  2165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:11:53.019132  2165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.60:34327: Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
I20260925 13:11:53.033603   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:53.033849   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:53.036959   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:34327
I20260925 13:11:53.037034  2230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:34327 every 8 connection(s)
I20260925 13:11:53.037226   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:53.037386  2231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.038082  2231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:53.041970  2231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:53.045012  2099 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } attempt: 1
I20260925 13:11:53.045414  2231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:53.048926  2099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:53.050017  2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap starting.
I20260925 13:11:53.050475  2231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.051137  2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: No bootstrap required, opened a new log
I20260925 13:11:53.051287  2231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.051359  2231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.051384  2231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1dfdc1200b406fa843ce340fbeaf09, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.051440  2231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.051671  2231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:53.051656  2236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.051796  2236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.052260  2165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } attempt: 1
I20260925 13:11:53.052668  2236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:53.052727  2236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.052860  2236 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
W20260925 13:11:53.053340  2247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:53.053429  2247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:53.053512  2099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: Bootstrap starting.
I20260925 13:11:53.054010  2099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.054627  2099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: No bootstrap required, opened a new log
I20260925 13:11:53.054768  2099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.054847  2099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.054872  2099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b61637d5b2d427980a42ef58d9f7221, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.054927  2099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.055130  2099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:53.055397  2249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.055480  2249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.056366   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260925 13:11:53.056442  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
W20260925 13:11:53.056501  2260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 13:11:53.056523  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 0.
W20260925 13:11:53.056565  2260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:53.056579  2165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Bootstrap starting.
I20260925 13:11:53.056674  2177 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: 
I20260925 13:11:53.056990  2165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.057143  2236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:53.057188  2236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:53.057214  2236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.057631  2165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: No bootstrap required, opened a new log
I20260925 13:11:53.057796  2165 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.057876  2165 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.057910  2165 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8c0be10d6644cd822c356e727e40fd, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.057958  2236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.057966  2165 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.058117  2236 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 1 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
I20260925 13:11:53.058180  2165 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:53.058238  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:53.058254  2139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd" is_pre_election: true
I20260925 13:11:53.058319  2073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.058339  2139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 0.
I20260925 13:11:53.058867  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 1.
I20260925 13:11:53.059036  2177 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: 
I20260925 13:11:53.059139  2236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:53.059270  2236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 LEADER]: Becoming Leader. State: Replica: 5c1dfdc1200b406fa843ce340fbeaf09, State: Running, Role: LEADER
I20260925 13:11:53.059340  2236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:53.059564   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260925 13:11:53.059576  2139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd"
W20260925 13:11:53.059638  2274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 13:11:53.059695  2139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.059835   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:53.060431  2139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 1.
I20260925 13:11:53.059587  2270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.060691  2270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: LEADER
W20260925 13:11:53.059691  2274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:53.060904  2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.060969  2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.060985  2280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:53.061225  2280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260925 13:11:53.061586  2277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.061728  2276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.061775  2279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:53.061925  2139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Refusing update from remote peer 5c1dfdc1200b406fa843ce340fbeaf09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.061965  2073 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Refusing update from remote peer 5c1dfdc1200b406fa843ce340fbeaf09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.062095  2236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.062162  2270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.062330   392 server_base.cc:1034] running on GCE node
I20260925 13:11:53.062484   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:53.062520   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:53.062533   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341913062534 us; error 0 us; skew 500 ppm
I20260925 13:11:53.062958  2249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.063055  2249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.063021  2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.063124  2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.063287   392 webserver.cc:460] Webserver started at http://127.0.98.1:40585/ using document root <none> and password file <none>
I20260925 13:11:53.063364   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:53.063405   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:53.063438  2236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.063537  2236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:53.063447   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:53.063840  2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.063931  2282 mvcc.cc:205] Tried to move back new op lower bound from 7333240475900022784 to 7333240475891560448. Current Snapshot: MvccSnapshot[applied={T|T < 7333240475900022784}]
I20260925 13:11:53.063901  2270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.064020  2270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:53.063876   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "919cdc37e0f843f7b2dac0b863a27e22"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.063987  2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.064100  2249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:53.064179  2280 catalog_manager.cc:1347] Generated new cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:53.064217  2249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:53.064224  2280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:53.065258   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:53.065838  2293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.065969   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:53.066015   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "919cdc37e0f843f7b2dac0b863a27e22"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.066051   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:53.080088  2280 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:53.080698  2280 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:53.088672  2280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Generated new TSK 0
I20260925 13:11:53.088820  2280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:53.090484   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:53.090763   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:53.090982   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:53.091199   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:53.091229   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.091250   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:53.091262   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.094754   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:43385
I20260925 13:11:53.094828  2357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:43385 every 8 connection(s)
I20260925 13:11:53.095242   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:53.097162  2366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.097250  2368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.097174  2365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:53.097734   392 server_base.cc:1034] running on GCE node
I20260925 13:11:53.097893   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:53.097961   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:53.098021   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341913098020 us; error 0 us; skew 500 ppm
I20260925 13:11:53.099006   392 webserver.cc:460] Webserver started at http://127.0.98.2:38333/ using document root <none> and password file <none>
I20260925 13:11:53.099133   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:53.099184   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:53.099243   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:53.099563   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "a75e127731684428a5e320ef5a358a3e"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.099728  2359 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40169
I20260925 13:11:53.099792  2359 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.099879  2359 heartbeater.cc:511] Master 127.0.98.62:40169 requested a full tablet report, sending...
I20260925 13:11:53.100284  2358 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:53.100308  2063 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:53.100329  2358 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.100519  2358 heartbeater.cc:511] Master 127.0.98.60:34327 requested a full tablet report, sending...
I20260925 13:11:53.100731   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:53.100822  2195 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:53.100869  2360 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:53.100917  2360 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.100986  2360 heartbeater.cc:511] Master 127.0.98.61:46353 requested a full tablet report, sending...
I20260925 13:11:53.101276  2129 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:53.101449  2373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.101537  2195 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42916
I20260925 13:11:53.101595   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.101649   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "a75e127731684428a5e320ef5a358a3e"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.101696   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:53.112497   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:53.112766   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:53.112938   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:53.113202   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:53.113247   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.113287   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:53.113314   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.117043   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:40761
I20260925 13:11:53.117194  2436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:40761 every 8 connection(s)
I20260925 13:11:53.117638   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:53.119170  2443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.119690  2447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:53.120164  2445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:53.120179   392 server_base.cc:1034] running on GCE node
I20260925 13:11:53.120415   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:53.120445   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:53.120462   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341913120462 us; error 0 us; skew 500 ppm
I20260925 13:11:53.121279   392 webserver.cc:460] Webserver started at http://127.0.98.3:39111/ using document root <none> and password file <none>
I20260925 13:11:53.121363   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:53.121407   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:53.121443   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:53.121752   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-2-root/instance:
uuid: "0b2184873e294eb78e5d9ad4f879fe03"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.121861  2438 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40169
I20260925 13:11:53.121914  2438 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.121984  2438 heartbeater.cc:511] Master 127.0.98.62:40169 requested a full tablet report, sending...
I20260925 13:11:53.122287  2063 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.122493  2437 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:53.122539  2437 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.122617  2437 heartbeater.cc:511] Master 127.0.98.60:34327 requested a full tablet report, sending...
I20260925 13:11:53.122929  2439 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:53.122942  2195 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.122990  2439 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.123085  2439 heartbeater.cc:511] Master 127.0.98.61:46353 requested a full tablet report, sending...
I20260925 13:11:53.123217   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:53.123339  2129 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.123452  2195 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42928
I20260925 13:11:53.123776  2452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.123974   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.124044   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-2-root
uuid: "0b2184873e294eb78e5d9ad4f879fe03"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:53.124089   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:53.141609   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:53.141882   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:53.142032   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:53.142303   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:53.142339   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.142387   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:53.142418   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.146694   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:46603
I20260925 13:11:53.146788  2515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:46603 every 8 connection(s)
I20260925 13:11:53.151499  2518 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:53.151583  2518 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.151670  2518 heartbeater.cc:511] Master 127.0.98.61:46353 requested a full tablet report, sending...
I20260925 13:11:53.151971  2129 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.152189  2516 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:53.152271  2516 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.152349  2516 heartbeater.cc:511] Master 127.0.98.60:34327 requested a full tablet report, sending...
I20260925 13:11:53.152379  2517 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40169
I20260925 13:11:53.152467  2517 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:53.152638  2517 heartbeater.cc:511] Master 127.0.98.62:40169 requested a full tablet report, sending...
I20260925 13:11:53.152665  2195 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.153129  2063 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.153429  2195 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42930
I20260925 13:11:53.153678   392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006626907s
I20260925 13:11:53.162789  2195 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42932:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  hash_schema {
    columns {
      name: "key"
    }
    num_buckets: 3
    seed: 0
  }
}
W20260925 13:11:53.162974  2195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:53.170445  2322 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0fd9861974514a375b7469364c65e (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.170540  2320 tablet_service.cc:1467] Processing CreateTablet for tablet 077b55ad10884161a8f316758fc55be1 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.170562  2399 tablet_service.cc:1467] Processing CreateTablet for tablet 077b55ad10884161a8f316758fc55be1 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.170607  2400 tablet_service.cc:1467] Processing CreateTablet for tablet 4035085c2f334a30b294ae2e8184a767 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.170644  2320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077b55ad10884161a8f316758fc55be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.170712  2400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4035085c2f334a30b294ae2e8184a767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.170809  2399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077b55ad10884161a8f316758fc55be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.170562  2401 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0fd9861974514a375b7469364c65e (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.170920  2322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0fd9861974514a375b7469364c65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.170924  2401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0fd9861974514a375b7469364c65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.170467  2321 tablet_service.cc:1467] Processing CreateTablet for tablet 4035085c2f334a30b294ae2e8184a767 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.172112  2536 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:53.172215  2321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4035085c2f334a30b294ae2e8184a767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.173949  2480 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0fd9861974514a375b7469364c65e (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.173962  2478 tablet_service.cc:1467] Processing CreateTablet for tablet 077b55ad10884161a8f316758fc55be1 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.174081  2480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0fd9861974514a375b7469364c65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.174201  2478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077b55ad10884161a8f316758fc55be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.175195  2479 tablet_service.cc:1467] Processing CreateTablet for tablet 4035085c2f334a30b294ae2e8184a767 (DEFAULT_TABLE table=test-table [id=97af554488bd49f798114e47e4ceba3d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:53.175325  2479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4035085c2f334a30b294ae2e8184a767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:53.175354  2537 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: Bootstrap starting.
I20260925 13:11:53.175815  2537 tablet_bootstrap.cc:654] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.189312  2536 tablet_bootstrap.cc:654] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.189802  2537 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: No bootstrap required, opened a new log
I20260925 13:11:53.189877  2537 ts_tablet_manager.cc:1397] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: Time spent bootstrapping tablet: real 0.015s	user 0.001s	sys 0.000s
I20260925 13:11:53.190111  2537 raft_consensus.cc:348] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.190243  2537 raft_consensus.cc:374] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.190270  2537 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.190325  2537 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.190482  2537 ts_tablet_manager.cc:1428] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:53.190271  2536 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: No bootstrap required, opened a new log
I20260925 13:11:53.190580  2537 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: Bootstrap starting.
I20260925 13:11:53.190619  2536 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.019s	user 0.002s	sys 0.000s
I20260925 13:11:53.190784  2536 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.190850  2536 raft_consensus.cc:374] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.190872  2536 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.190910  2536 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.191032  2537 tablet_bootstrap.cc:654] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.191071  2536 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.191174  2536 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:53.191344  2516 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:53.191573  2536 tablet_bootstrap.cc:654] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.192389  2358 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:53.193178  2536 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: No bootstrap required, opened a new log
I20260925 13:11:53.193230  2536 ts_tablet_manager.cc:1397] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:53.193375  2536 raft_consensus.cc:348] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.193437  2536 raft_consensus.cc:374] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.193521  2536 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.193565  2536 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.193722  2536 ts_tablet_manager.cc:1428] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.193437  2541 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:53.194142  2537 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: No bootstrap required, opened a new log
I20260925 13:11:53.194195  2537 ts_tablet_manager.cc:1397] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: Time spent bootstrapping tablet: real 0.004s	user 0.001s	sys 0.000s
I20260925 13:11:53.194209  2536 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:53.194339  2537 raft_consensus.cc:348] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.194396  2537 raft_consensus.cc:374] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.194422  2537 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.194468  2537 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.194514  2541 tablet_bootstrap.cc:654] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.194550  2537 ts_tablet_manager.cc:1428] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.194626  2537 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Bootstrap starting.
I20260925 13:11:53.194643  2536 tablet_bootstrap.cc:654] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.195010  2537 tablet_bootstrap.cc:654] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.195395  2536 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: No bootstrap required, opened a new log
I20260925 13:11:53.195399  2541 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: No bootstrap required, opened a new log
I20260925 13:11:53.195456  2536 ts_tablet_manager.cc:1397] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:53.195467  2541 ts_tablet_manager.cc:1397] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:53.195616  2541 raft_consensus.cc:348] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.195701  2541 raft_consensus.cc:374] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.195722  2541 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.195762  2541 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.195931  2541 ts_tablet_manager.cc:1428] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.196022  2541 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:53.195614  2536 raft_consensus.cc:348] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.196146  2536 raft_consensus.cc:374] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.196169  2536 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.196238  2536 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.196321  2536 ts_tablet_manager.cc:1428] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:53.196170  2437 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:53.196542  2537 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: No bootstrap required, opened a new log
I20260925 13:11:53.196588  2537 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:53.196636  2541 tablet_bootstrap.cc:654] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.196719  2537 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.196784  2537 raft_consensus.cc:374] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.196813  2537 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.196851  2537 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.196923  2537 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.198007  2541 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: No bootstrap required, opened a new log
I20260925 13:11:53.198073  2541 ts_tablet_manager.cc:1397] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:11:53.198230  2541 raft_consensus.cc:348] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.198267  2541 raft_consensus.cc:374] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.198280  2541 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.198333  2541 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.198421  2541 ts_tablet_manager.cc:1428] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.198522  2541 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:53.198529  2540 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:53.198611  2540 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.198781  2540 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.198961  2541 tablet_bootstrap.cc:654] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:53.199566  2541 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: No bootstrap required, opened a new log
I20260925 13:11:53.199615  2541 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:53.199755  2541 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.199813  2541 raft_consensus.cc:374] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:53.199841  2541 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: FOLLOWER
I20260925 13:11:53.199900  2541 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.199987  2541 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:53.202571  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:53.202631  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.202663  2332 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 0.
I20260925 13:11:53.202728  2411 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 0.
I20260925 13:11:53.202858  2454 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: 
I20260925 13:11:53.203018  2540 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:53.203073  2540 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:53.203102  2540 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.203989  2540 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.204120  2540 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 1 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.204368  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:53.204447  2332 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.204481  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:53.204603  2411 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.205161  2332 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 1.
I20260925 13:11:53.205341  2454 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: 
I20260925 13:11:53.205456  2540 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:53.205451  2411 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 1.
I20260925 13:11:53.205631  2540 raft_consensus.cc:686] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 LEADER]: Becoming Leader. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:53.205699  2540 consensus_queue.cc:237] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.206238  2193 catalog_manager.cc:5495] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3). New cstate: current_term: 1 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:53.222965  2545 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:53.223117  2545 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.223456  2545 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.227129  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.227077  2490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:53.227214  2411 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 0.
I20260925 13:11:53.227223  2490 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 0.
I20260925 13:11:53.227370  2295 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: 
I20260925 13:11:53.227494  2545 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:53.227540  2545 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:53.227587  2545 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.228231  2545 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.228363  2545 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 election: Requested vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.228551  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:53.228554  2490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:53.228667  2411 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.228674  2490 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.229339  2411 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 1.
I20260925 13:11:53.229516  2490 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 1.
I20260925 13:11:53.229552  2295 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:53.229646  2545 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:53.229763  2545 raft_consensus.cc:686] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Becoming Leader. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Running, Role: LEADER
I20260925 13:11:53.229815  2545 consensus_queue.cc:237] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.230471  2193 catalog_manager.cc:5495] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1). New cstate: current_term: 1 leader_uuid: "919cdc37e0f843f7b2dac0b863a27e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:53.244095  2545 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:53.244184  2545 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.244349  2545 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.244624  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.244685  2490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:53.244766  2411 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 0.
I20260925 13:11:53.244778  2490 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 0.
I20260925 13:11:53.244916  2295 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: 
I20260925 13:11:53.245118  2545 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:53.245160  2545 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:53.245184  2545 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.245780  2545 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.245903  2545 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 election: Requested vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.246043  2411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:53.246100  2411 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.246070  2490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:53.246140  2490 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:53.246636  2411 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 1.
I20260925 13:11:53.246790  2295 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:53.246778  2490 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 1.
I20260925 13:11:53.246920  2545 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:53.246980  2545 raft_consensus.cc:686] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Becoming Leader. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Running, Role: LEADER
I20260925 13:11:53.247038  2545 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.247547  2193 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1). New cstate: current_term: 1 leader_uuid: "919cdc37e0f843f7b2dac0b863a27e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:53.273368  2411 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.273324  2490 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.273636  2489 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.273674  2542 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.273842  2411 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.274034  2332 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.274055  2411 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:53.274219  2552 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.274287  2540 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.274868  2545 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.274968  2545 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.275058  2545 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:53.303578  2332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c0fd9861974514a375b7469364c65e"
dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
 from {username='slave'} at 127.0.0.1:48834
I20260925 13:11:53.303699  2332 raft_consensus.cc:580] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Received request to step down
I20260925 13:11:53.303720  2332 raft_consensus.cc:3032] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:53.303732  2332 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Running, Role: LEADER
I20260925 13:11:53.303822  2332 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.303958  2332 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.478673  2540 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 919cdc37e0f843f7b2dac0b863a27e22)
I20260925 13:11:53.478813  2540 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.479012  2540 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:53.479271  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.479291  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:53.479405  2410 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 1.
I20260925 13:11:53.479485  2332 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 2.
I20260925 13:11:53.479662  2454 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:53.479799  2540 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:11:53.479839  2540 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 919cdc37e0f843f7b2dac0b863a27e22)
I20260925 13:11:53.479872  2540 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.480545  2540 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.480687  2540 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 election: Requested vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:53.480830  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:53.480894  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:53.481022  2410 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.481794  2332 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 2.
I20260925 13:11:53.481954  2454 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: 
I20260925 13:11:53.481801  2410 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 2.
I20260925 13:11:53.482018  2540 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:11:53.482146  2540 raft_consensus.cc:686] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 LEADER]: Becoming Leader. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:53.482198  2540 consensus_queue.cc:237] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.482564  2195 catalog_manager.cc:5495] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: term changed from 1 to 2, leader changed from 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1) to 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3). New cstate: current_term: 2 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:53.505247  2332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "077b55ad10884161a8f316758fc55be1"
dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
 from {username='slave'} at 127.0.0.1:48834
I20260925 13:11:53.505373  2332 raft_consensus.cc:580] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Received request to step down
I20260925 13:11:53.505401  2332 raft_consensus.cc:3032] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:53.505422  2332 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Running, Role: LEADER
I20260925 13:11:53.505481  2332 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.505564  2332 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.567500  2332 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260925 13:11:53.567829  2540 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:53.569290  2410 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260925 13:11:53.569599  2540 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:53.655158  2549 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 919cdc37e0f843f7b2dac0b863a27e22)
I20260925 13:11:53.655279  2549 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.655503  2549 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.659296  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:53.659431  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:53.659431  2332 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 2.
I20260925 13:11:53.659644  2375 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:53.659739  2549 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:11:53.659804  2549 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 919cdc37e0f843f7b2dac0b863a27e22)
I20260925 13:11:53.659832  2549 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.660524  2549 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.660671  2549 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:53.660825  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:53.660821  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:53.661521  2332 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 2.
I20260925 13:11:53.661720  2375 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 0b2184873e294eb78e5d9ad4f879fe03
I20260925 13:11:53.661870  2549 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:11:53.661990  2549 raft_consensus.cc:686] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Becoming Leader. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Running, Role: LEADER
I20260925 13:11:53.662053  2549 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.662526  2195 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e reported cstate change: term changed from 1 to 2, leader changed from 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1) to a75e127731684428a5e320ef5a358a3e (127.0.98.2). New cstate: current_term: 2 leader_uuid: "a75e127731684428a5e320ef5a358a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:53.703150  2585 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 919cdc37e0f843f7b2dac0b863a27e22)
I20260925 13:11:53.703289  2585 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.703425  2585 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.703734  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.703792  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:53.703914  2332 raft_consensus.cc:2379] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b2184873e294eb78e5d9ad4f879fe03 in current term 2: Already voted for candidate a75e127731684428a5e320ef5a358a3e in this term.
I20260925 13:11:53.704061  2454 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03; no voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e
I20260925 13:11:53.704195  2585 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:53.704933  2585 raft_consensus.cc:2726] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:11:53.709733  2410 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "077b55ad10884161a8f316758fc55be1"
dest_uuid: "a75e127731684428a5e320ef5a358a3e"
 from {username='slave'} at 127.0.0.1:48656
I20260925 13:11:53.709805  2410 raft_consensus.cc:580] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Received request to step down
I20260925 13:11:53.709822  2410 raft_consensus.cc:3032] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:53.709863  2410 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Running, Role: LEADER
I20260925 13:11:53.709928  2410 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.710081  2410 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:53.914412  2489 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c0fd9861974514a375b7469364c65e"
dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
 from {username='slave'} at 127.0.0.1:49990
I20260925 13:11:53.914551  2489 raft_consensus.cc:580] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 LEADER]: Received request to step down
I20260925 13:11:53.914589  2489 raft_consensus.cc:3032] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:11:53.914618  2489 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:53.914682  2489 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.914789  2489 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:53.934051  2585 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:53.934165  2585 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.934376  2585 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.934688  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:53.934701  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:53.934844  2410 raft_consensus.cc:2396] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b2184873e294eb78e5d9ad4f879fe03 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260925 13:11:53.934846  2332 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 2.
I20260925 13:11:53.935063  2454 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: a75e127731684428a5e320ef5a358a3e
I20260925 13:11:53.935252  2585 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:11:53.935298  2585 raft_consensus.cc:482] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:53.935317  2585 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:53.935971  2585 raft_consensus.cc:504] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.936097  2585 leader_election.cc:288] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:53.936257  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:53.936329  2410 raft_consensus.cc:2396] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b2184873e294eb78e5d9ad4f879fe03 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260925 13:11:53.936275  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077b55ad10884161a8f316758fc55be1" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:53.936359  2332 raft_consensus.cc:3037] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:53.936961  2332 raft_consensus.cc:2454] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 3.
I20260925 13:11:53.937239  2454 leader_election.cc:302] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, 919cdc37e0f843f7b2dac0b863a27e22; no voters: a75e127731684428a5e320ef5a358a3e
I20260925 13:11:53.937394  2585 raft_consensus.cc:2781] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:11:53.937536  2585 raft_consensus.cc:686] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Becoming Leader. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:53.937598  2585 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:53.938022  2188 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: term changed from 2 to 3, leader changed from a75e127731684428a5e320ef5a358a3e (127.0.98.2) to 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3). New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:54.009631  2549 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.009788  2549 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.010133  2549 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.010358  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:54.010403  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:54.010525  2489 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 3.
I20260925 13:11:54.010695  2375 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:54.010788  2549 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:11:54.010847  2549 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.010879  2549 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:54.011921  2549 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.012162  2549 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 3 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.012311  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:54.012328  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:54.012975  2489 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 3.
I20260925 13:11:54.013228  2375 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:54.013347  2549 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:11:54.013407  2549 raft_consensus.cc:686] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 LEADER]: Becoming Leader. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Running, Role: LEADER
I20260925 13:11:54.013468  2549 consensus_queue.cc:237] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.013878  2188 catalog_manager.cc:5495] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e reported cstate change: term changed from 2 to 3, leader changed from 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3) to a75e127731684428a5e320ef5a358a3e (127.0.98.2). New cstate: current_term: 3 leader_uuid: "a75e127731684428a5e320ef5a358a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:54.045666  2332 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260925 13:11:54.045964  2585 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:54.047209  2410 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260925 13:11:54.047281  2410 pending_rounds.cc:76] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Aborting all ops after (but not including) 2
I20260925 13:11:54.047319  2410 pending_rounds.cc:97] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260925 13:11:54.047359  2410 raft_consensus.cc:2866] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 13:11:54.047570  2585 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:54.054482  2542 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.054561  2542 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.054740  2542 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.054941  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:54.055069  2489 raft_consensus.cc:2379] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 919cdc37e0f843f7b2dac0b863a27e22 in current term 3: Already voted for candidate a75e127731684428a5e320ef5a358a3e in this term.
I20260925 13:11:54.054939  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:54.055260  2295 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22; no voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e
I20260925 13:11:54.055358  2542 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:54.056177  2542 raft_consensus.cc:2726] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 13:11:54.056951  2260 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:54.056996  2260 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: loading cluster ID for follower catalog manager: success
I20260925 13:11:54.057822  2260 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: acquiring CA information for follower catalog manager: success
I20260925 13:11:54.058084  2260 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:54.061275  2274 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:54.061322  2274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: loading cluster ID for follower catalog manager: success
I20260925 13:11:54.062011  2274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: acquiring CA information for follower catalog manager: success
I20260925 13:11:54.062211  2274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:54.107452  2489 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Refusing update from remote peer a75e127731684428a5e320ef5a358a3e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260925 13:11:54.107750  2549 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:54.108914  2332 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer a75e127731684428a5e320ef5a358a3e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260925 13:11:54.109237  2584 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:54.115717  2489 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767"
dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
 from {username='slave'} at 127.0.0.1:49990
I20260925 13:11:54.115818  2489 raft_consensus.cc:580] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 LEADER]: Received request to step down
I20260925 13:11:54.115849  2489 raft_consensus.cc:3032] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:11:54.115869  2489 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:54.115917  2489 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.116007  2489 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:54.232070  2584 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.232177  2584 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.232335  2584 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.232556  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:54.232582  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:54.232805  2489 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 2.
I20260925 13:11:54.232992  2375 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:54.233103  2584 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:11:54.233139  2584 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.233170  2584 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:54.233865  2584 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.233987  2584 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.234164  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:54.234194  2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "a75e127731684428a5e320ef5a358a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:54.234751  2489 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75e127731684428a5e320ef5a358a3e in term 2.
I20260925 13:11:54.234905  2375 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:54.234997  2584 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:11:54.235042  2584 raft_consensus.cc:686] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Becoming Leader. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Running, Role: LEADER
I20260925 13:11:54.235085  2584 consensus_queue.cc:237] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.235600  2188 catalog_manager.cc:5495] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e reported cstate change: term changed from 1 to 2, leader changed from 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3) to a75e127731684428a5e320ef5a358a3e (127.0.98.2). New cstate: current_term: 2 leader_uuid: "a75e127731684428a5e320ef5a358a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:54.296126  2586 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b2184873e294eb78e5d9ad4f879fe03)
I20260925 13:11:54.296258  2586 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:54.296480  2586 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.296700  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:54.296726  2410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:54.296808  2489 raft_consensus.cc:2379] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 919cdc37e0f843f7b2dac0b863a27e22 in current term 2: Already voted for candidate a75e127731684428a5e320ef5a358a3e in this term.
I20260925 13:11:54.296979  2295 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22; no voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e
I20260925 13:11:54.297119  2586 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:54.297850  2586 raft_consensus.cc:2726] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:11:54.318540  2188 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42934:
table { table_id: "97af554488bd49f798114e47e4ceba3d" table_name: "test-table" } new_table_name: "NewTableName"
I20260925 13:11:54.320784   392 master.cc:537] Master@127.0.98.60:34327 shutting down...
I20260925 13:11:54.321483  2588 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.321931  2590 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.322270  2589 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.322911  2490 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Refusing update from remote peer a75e127731684428a5e320ef5a358a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260925 13:11:54.323033  2331 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Refusing update from remote peer a75e127731684428a5e320ef5a358a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260925 13:11:54.323192  2549 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:54.323555  2584 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:54.324229  2589 tablet.cc:1722] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.324900  2565 mvcc.cc:205] Tried to move back new op lower bound from 7333240481064833024 to 7333240480707817472. Current Snapshot: MvccSnapshot[applied={T|T < 7333240481064833024}]
I20260925 13:11:54.325603  2588 tablet.cc:1722] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.326563  2589 tablet.cc:1722] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.326776  2588 tablet.cc:1722] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.327431  2590 tablet.cc:1722] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.329166  2590 tablet.cc:1722] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:54.330065   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:54.330149   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:54.330194   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: stopping tablet replica
I20260925 13:11:54.332921   392 master.cc:552] Master@127.0.98.60:34327 shutdown complete.
I20260925 13:11:54.520910  2600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:54.521025  2600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.521160  2601 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:54.521281  2600 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.521270  2601 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.521478  2139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd" is_pre_election: true
I20260925 13:11:54.521566  2139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b61637d5b2d427980a42ef58d9f7221 in term 1.
I20260925 13:11:54.521648  2601 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.521682  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:54.521754  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 1.
I20260925 13:11:54.521759  2050 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:54.521908  2600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:11:54.521951  2600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:54.521975  2600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:54.522073  2112 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:54.522161  2601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:11:54.522195  2601 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:54.522225  2601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 1 FOLLOWER]: Advancing to term 2
W20260925 13:11:54.522256  2114 leader_election.cc:334] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
W20260925 13:11:54.522404  2049 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
I20260925 13:11:54.522976  2600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.522981  2601 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.523146  2601 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.523146  2600 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 election: Requested vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.523384  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:54.523417  2139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd"
I20260925 13:11:54.523489  2139 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b61637d5b2d427980a42ef58d9f7221 in current term 2: Already voted for candidate fd8c0be10d6644cd822c356e727e40fd in this term.
I20260925 13:11:54.523468  2073 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fd8c0be10d6644cd822c356e727e40fd in current term 2: Already voted for candidate 6b61637d5b2d427980a42ef58d9f7221 in this term.
W20260925 13:11:54.523648  2049 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
W20260925 13:11:54.523666  2114 leader_election.cc:334] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
I20260925 13:11:54.523697  2049 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221; no voters: 5c1dfdc1200b406fa843ce340fbeaf09, fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:54.523734  2114 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd8c0be10d6644cd822c356e727e40fd; no voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221
I20260925 13:11:54.523770  2600 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 13:11:54.523810  2601 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 13:11:54.534358   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:54.535908  2607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:54.536005  2606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:54.536065   392 server_base.cc:1034] running on GCE node
W20260925 13:11:54.536197  2609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:54.536370   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:54.536418   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:54.536432   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341914536433 us; error 0 us; skew 500 ppm
I20260925 13:11:54.537266   392 webserver.cc:460] Webserver started at http://127.0.98.60:34775/ using document root <none> and password file <none>
I20260925 13:11:54.537374   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:54.537408   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:54.537966   392 fs_manager.cc:709] Time spent opening directory manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:54.538494  2614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:54.538638   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:54.538689   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:54.538748   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:54.549386   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:54.549558   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:54.552014   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:34327
I20260925 13:11:54.552134  2666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:34327 every 8 connection(s)
I20260925 13:11:54.553336  2667 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:11:54.553784  2667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap starting.
I20260925 13:11:54.556222  2667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=7 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:54.556524  2667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap complete.
I20260925 13:11:54.556757  2667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.556849  2667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1dfdc1200b406fa843ce340fbeaf09, State: Initialized, Role: FOLLOWER
I20260925 13:11:54.556908  2667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.557150  2667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:54.557132  2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.557221  2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:54.558422  2681 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:54.558490  2681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: loading cluster ID for follower catalog manager: success
I20260925 13:11:54.559319  2681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: acquiring CA information for follower catalog manager: success
I20260925 13:11:54.559508  2681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:54.635113  2437 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:54.635351  2358 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:54.635914  2631 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a75e127731684428a5e320ef5a358a3e" instance_seqno: 1790341913116129) as {username='slave'} at 127.0.0.1:43038; Asking this server to re-register.
I20260925 13:11:54.635973  2630 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" instance_seqno: 1790341913093650) as {username='slave'} at 127.0.0.1:43054; Asking this server to re-register.
I20260925 13:11:54.636164  2437 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:54.636168  2516 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:54.636228  2358 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:54.636475  2631 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:54.636555  2629 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:54.636579  2630 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" instance_seqno: 1790341913145606) as {username='slave'} at 127.0.0.1:43060; Asking this server to re-register.
I20260925 13:11:54.636842  2516 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:54.637125  2630 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:54.773849  2601 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:54.774047  2601 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.774240  2601 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.774471  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:54.774557  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 2.
I20260925 13:11:54.774729  2112 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:54.774819  2601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:11:54.774847  2601 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:54.774875  2601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:54.775820  2601 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.776000  2601 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 3 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:54.776119  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:54.776213  2073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:54.776973  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 3.
I20260925 13:11:54.777258  2112 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:54.777359  2601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:11:54.777501  2601 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 LEADER]: Becoming Leader. State: Replica: fd8c0be10d6644cd822c356e727e40fd, State: Running, Role: LEADER
I20260925 13:11:54.777555  2601 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:54.777755  2685 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.777825  2685 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:54.778023  2686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:54.778156  2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" is_pre_election: true
I20260925 13:11:54.778244  2641 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 1.
I20260925 13:11:54.778172  2640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
I20260925 13:11:54.778301  2640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 1 FOLLOWER]: Advancing to term 3
I20260925 13:11:54.778316  2686 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=97af554488bd49f798114e47e4ceba3d]
I20260925 13:11:54.778478  2686 tablet_loader.cc:96] loaded metadata for tablet 06c0fd9861974514a375b7469364c65e (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:54.778522  2686 tablet_loader.cc:96] loaded metadata for tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:54.778575  2686 tablet_loader.cc:96] loaded metadata for tablet 4035085c2f334a30b294ae2e8184a767 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:54.778609  2686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:54.778672  2686 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:54.778699  2686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:54.778872  2686 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:54.778939  2686 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Loaded TSK: 0
I20260925 13:11:54.779075  2640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 3.
I20260925 13:11:54.779098  2686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:54.871994  2640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Refusing update from remote peer fd8c0be10d6644cd822c356e727e40fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20260925 13:11:54.872303  2685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260925 13:11:54.872838  2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.872920  2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:54.873389  2601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.873487  2601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:54.873520  2073 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Refusing update from remote peer fd8c0be10d6644cd822c356e727e40fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20260925 13:11:54.873698  2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.873769  2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:54.873806  2601 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260925 13:11:54.874285  2600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.874361  2600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:54.874322  2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.874408  2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:54.955042  2439 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:54.955169  2360 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:54.955322  2518 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:54.956321  2685 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:54.956409  2685 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:54.957121  2129 catalog_manager.cc:6040] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: NewTableName [id=97af554488bd49f798114e47e4ceba3d] alter complete (version 1)
I20260925 13:11:54.983136   392 master.cc:537] Master@127.0.98.61:46353 shutting down...
I20260925 13:11:54.985737   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:54.985852   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:54.986068   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: stopping tablet replica
I20260925 13:11:54.999002   392 master.cc:552] Master@127.0.98.61:46353 shutdown complete.
I20260925 13:11:55.157331  2670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fd8c0be10d6644cd822c356e727e40fd)
I20260925 13:11:55.157449  2691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fd8c0be10d6644cd822c356e727e40fd)
I20260925 13:11:55.157451  2670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.157599  2691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.157702  2691 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.157735  2670 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
W20260925 13:11:55.158511  2050 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353): Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
W20260925 13:11:55.159045  2618 leader_election.cc:334] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353): Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
I20260925 13:11:55.161391  2640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" is_pre_election: true
I20260925 13:11:55.161530  2640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b61637d5b2d427980a42ef58d9f7221 in term 3.
I20260925 13:11:55.161580  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:55.161670  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 3.
I20260925 13:11:55.161713  2049 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:55.161775  2615 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:55.161789  2691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 13:11:55.161821  2691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fd8c0be10d6644cd822c356e727e40fd)
I20260925 13:11:55.161834  2670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 13:11:55.161840  2691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:55.161890  2670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fd8c0be10d6644cd822c356e727e40fd)
I20260925 13:11:55.161934  2670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:55.162643  2670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.162674  2691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.162794  2670 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
I20260925 13:11:55.162818  2691 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 election: Requested vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.162902  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:55.162957  2073 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c1dfdc1200b406fa843ce340fbeaf09 in current term 4: Already voted for candidate 6b61637d5b2d427980a42ef58d9f7221 in this term.
I20260925 13:11:55.163151  2640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
I20260925 13:11:55.163223  2640 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b61637d5b2d427980a42ef58d9f7221 in current term 4: Already voted for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in this term.
W20260925 13:11:55.163321  2618 leader_election.cc:334] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353): Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
W20260925 13:11:55.163370  2050 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353): Network error: Client connection negotiation failed: client connection to 127.0.98.61:46353: connect: Connection refused (error 111)
I20260925 13:11:55.163424  2050 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221; no voters: 5c1dfdc1200b406fa843ce340fbeaf09, fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:55.163523  2691 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260925 13:11:55.163371  2618 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09; no voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:55.163683  2670 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260925 13:11:55.200675   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:55.202672  2703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:55.202800  2706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:55.202910  2704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:55.203007   392 server_base.cc:1034] running on GCE node
I20260925 13:11:55.203173   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:55.203229   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:55.203251   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341915203250 us; error 0 us; skew 500 ppm
I20260925 13:11:55.212777   392 webserver.cc:460] Webserver started at http://127.0.98.61:42569/ using document root <none> and password file <none>
I20260925 13:11:55.213042   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:55.213127   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:55.213829   392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:55.214526  2711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:55.214874   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:55.214946   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
uuid: "fd8c0be10d6644cd822c356e727e40fd"
format_stamp: "Formatted at 2026-09-25 13:11:52 on dist-test-slave-2538"
I20260925 13:11:55.215001   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:55.222473   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:55.222718   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:55.225793   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:46353
I20260925 13:11:55.225843  2763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:46353 every 8 connection(s)
I20260925 13:11:55.226557  2764 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:11:55.226883  2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Bootstrap starting.
I20260925 13:11:55.228807  2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:55.229007  2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Bootstrap complete.
I20260925 13:11:55.229251  2764 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.229324  2764 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8c0be10d6644cd822c356e727e40fd, State: Initialized, Role: FOLLOWER
I20260925 13:11:55.229385  2764 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.229547  2764 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:55.229578  2767 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.229698  2767 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.230691  2778 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:55.230728  2778 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: loading cluster ID for follower catalog manager: success
I20260925 13:11:55.231372  2778 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: acquiring CA information for follower catalog manager: success
I20260925 13:11:55.231490  2778 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:55.263152  2439 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:55.263254  2518 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:55.263357  2360 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:55.263501  2728 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a75e127731684428a5e320ef5a358a3e" instance_seqno: 1790341913116129) as {username='slave'} at 127.0.0.1:59738; Asking this server to re-register.
I20260925 13:11:55.263613  2726 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" instance_seqno: 1790341913145606) as {username='slave'} at 127.0.0.1:59752; Asking this server to re-register.
I20260925 13:11:55.263633  2727 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" instance_seqno: 1790341913093650) as {username='slave'} at 127.0.0.1:59730; Asking this server to re-register.
I20260925 13:11:55.263770  2439 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.263819  2518 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.263832  2360 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.264047  2727 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:55.264091  2728 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:55.264173  2726 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:55.418193  2670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:55.418345  2670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.418586  2670 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
I20260925 13:11:55.418789  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:55.418908  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 4.
I20260925 13:11:55.419055  2615 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: 
I20260925 13:11:55.419181  2670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260925 13:11:55.419219  2670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:55.419241  2670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:55.420010  2670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.420158  2670 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 5 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353)
I20260925 13:11:55.420334  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:55.420480  2073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:11:55.421344  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 5.
I20260925 13:11:55.421612  2615 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221; no voters: 
I20260925 13:11:55.421722  2670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Leader election won for term 5
I20260925 13:11:55.421880  2670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 LEADER]: Becoming Leader. State: Replica: 5c1dfdc1200b406fa843ce340fbeaf09, State: Running, Role: LEADER
I20260925 13:11:55.421952  2670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.422163  2782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.422255  2782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:55.422540  2783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:55.422564  2737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd"
I20260925 13:11:55.422729  2737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 3 FOLLOWER]: Advancing to term 5
I20260925 13:11:55.422600  2738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd" is_pre_election: true
I20260925 13:11:55.422794  2783 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=97af554488bd49f798114e47e4ceba3d]
I20260925 13:11:55.422930  2783 tablet_loader.cc:96] loaded metadata for tablet 06c0fd9861974514a375b7469364c65e (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.423007  2783 tablet_loader.cc:96] loaded metadata for tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.423061  2783 tablet_loader.cc:96] loaded metadata for tablet 4035085c2f334a30b294ae2e8184a767 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.423106  2783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:55.423189  2783 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:55.423221  2783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:55.423416  2783 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:55.423507  2783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Loaded TSK: 0
I20260925 13:11:55.423616  2783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:55.423614  2737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1dfdc1200b406fa843ce340fbeaf09 in term 5.
I20260925 13:11:55.500656  2073 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Refusing update from remote peer 5c1dfdc1200b406fa843ce340fbeaf09: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 5 index: 18. (index mismatch)
I20260925 13:11:55.501116  2782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260925 13:11:55.501652  2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.501760  2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.502595  2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.502738  2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:55.502628  2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.502802  2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.502679  2737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Refusing update from remote peer 5c1dfdc1200b406fa843ce340fbeaf09: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 5 index: 18. (index mismatch)
I20260925 13:11:55.503029  2782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260925 13:11:55.503573  2788 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.503604  2767 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1dfdc1200b406fa843ce340fbeaf09. Latest consensus state: current_term: 5 leader_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.503700  2788 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.503728  2767 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.527765  2358 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:55.527830  2516 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:55.527766  2437 heartbeater.cc:503] Master 127.0.98.60:34327 was elected leader, sending a full tablet report...
I20260925 13:11:55.554855   392 master.cc:537] Master@127.0.98.60:34327 shutting down...
I20260925 13:11:55.557601   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 LEADER]: Raft consensus shutting down.
I20260925 13:11:55.557747   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:55.557822   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:55.559780   392 master.cc:552] Master@127.0.98.60:34327 shutdown complete.
I20260925 13:11:55.703684  2691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:55.703888  2767 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:55.703842  2691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.704044  2767 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.704144  2691 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.704293  2767 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
W20260925 13:11:55.705041  2714 leader_election.cc:334] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
W20260925 13:11:55.705166  2049 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
I20260925 13:11:55.707494  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:55.707588  2737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd" is_pre_election: true
I20260925 13:11:55.707607  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 5.
I20260925 13:11:55.707650  2737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b61637d5b2d427980a42ef58d9f7221 in term 5.
I20260925 13:11:55.707751  2712 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 5c1dfdc1200b406fa843ce340fbeaf09
I20260925 13:11:55.707762  2050 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 5c1dfdc1200b406fa843ce340fbeaf09
I20260925 13:11:55.707847  2767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260925 13:11:55.707885  2691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260925 13:11:55.707895  2767 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:55.707930  2691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c1dfdc1200b406fa843ce340fbeaf09)
I20260925 13:11:55.707965  2767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:55.707973  2691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:11:55.708771  2767 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.708771  2691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.708946  2691 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 election: Requested vote from peers fd8c0be10d6644cd822c356e727e40fd (127.0.98.61:46353), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.708945  2767 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.709132  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:55.709200  2073 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fd8c0be10d6644cd822c356e727e40fd in current term 6: Already voted for candidate 6b61637d5b2d427980a42ef58d9f7221 in this term.
I20260925 13:11:55.709177  2737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b61637d5b2d427980a42ef58d9f7221" candidate_term: 6 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "fd8c0be10d6644cd822c356e727e40fd"
I20260925 13:11:55.709256  2737 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b61637d5b2d427980a42ef58d9f7221 in current term 6: Already voted for candidate fd8c0be10d6644cd822c356e727e40fd in this term.
W20260925 13:11:55.709554  2714 leader_election.cc:334] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
W20260925 13:11:55.709554  2049 leader_election.cc:334] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327): Network error: Client connection negotiation failed: client connection to 127.0.98.60:34327: connect: Connection refused (error 111)
I20260925 13:11:55.709627  2714 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd8c0be10d6644cd822c356e727e40fd; no voters: 5c1dfdc1200b406fa843ce340fbeaf09, 6b61637d5b2d427980a42ef58d9f7221
I20260925 13:11:55.709649  2049 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221; no voters: 5c1dfdc1200b406fa843ce340fbeaf09, fd8c0be10d6644cd822c356e727e40fd
I20260925 13:11:55.709708  2691 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260925 13:11:55.709818  2767 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260925 13:11:55.761279   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:55.763021  2799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:55.763091  2796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:55.763144   392 server_base.cc:1034] running on GCE node
W20260925 13:11:55.763217  2797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:55.763438   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:55.763484   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:55.763499   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341915763498 us; error 0 us; skew 500 ppm
I20260925 13:11:55.764295   392 webserver.cc:460] Webserver started at http://127.0.98.60:34775/ using document root <none> and password file <none>
I20260925 13:11:55.764428   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:55.764482   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:55.765172   392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:55.765736  2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:55.765969   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:55.766034   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:55.766095   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:55.774338   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:55.774571   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:55.777473   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:34327
I20260925 13:11:55.777508  2856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:34327 every 8 connection(s)
I20260925 13:11:55.778268  2857 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:11:55.778590  2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap starting.
I20260925 13:11:55.780789  2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:55.781044  2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: Bootstrap complete.
I20260925 13:11:55.781380  2857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.781461  2857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1dfdc1200b406fa843ce340fbeaf09, State: Initialized, Role: FOLLOWER
I20260925 13:11:55.781497  2857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 5.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.781672  2857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:55.781674  2860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.781842  2860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:55.783046  2871 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:55.783118  2871 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: loading cluster ID for follower catalog manager: success
I20260925 13:11:55.783816  2871 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: acquiring CA information for follower catalog manager: success
I20260925 13:11:55.783943  2871 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:55.834548  2358 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:55.834662  2437 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:55.834864  2516 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:55.835075  2820 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a75e127731684428a5e320ef5a358a3e" instance_seqno: 1790341913116129) as {username='slave'} at 127.0.0.1:43272; Asking this server to re-register.
I20260925 13:11:55.835090  2821 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" instance_seqno: 1790341913093650) as {username='slave'} at 127.0.0.1:43268; Asking this server to re-register.
I20260925 13:11:55.835162  2819 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" instance_seqno: 1790341913145606) as {username='slave'} at 127.0.0.1:43252; Asking this server to re-register.
I20260925 13:11:55.835268  2437 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.835412  2516 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.835428  2358 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:55.835523  2819 ts_manager.cc:194] Registered new tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:55.835709  2819 ts_manager.cc:194] Registered new tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:55.835732  2821 ts_manager.cc:194] Registered new tserver with Master: 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:55.946964  2767 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:55.947091  2767 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.947316  2767 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.947579  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 7 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221" is_pre_election: true
I20260925 13:11:55.947672  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 6.
I20260925 13:11:55.947798  2712 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:55.947916  2767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260925 13:11:55.947976  2767 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:55.948009  2767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:55.948762  2767 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.948913  2767 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 7 election: Requested vote from peers 6b61637d5b2d427980a42ef58d9f7221 (127.0.98.62:40169), 5c1dfdc1200b406fa843ce340fbeaf09 (127.0.98.60:34327)
I20260925 13:11:55.949134  2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 7 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "6b61637d5b2d427980a42ef58d9f7221"
I20260925 13:11:55.949215  2073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:11:55.949860  2073 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 7.
I20260925 13:11:55.950033  2712 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b61637d5b2d427980a42ef58d9f7221, fd8c0be10d6644cd822c356e727e40fd; no voters: 
I20260925 13:11:55.950121  2767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 FOLLOWER]: Leader election won for term 7
I20260925 13:11:55.950187  2767 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 LEADER]: Becoming Leader. State: Replica: fd8c0be10d6644cd822c356e727e40fd, State: Running, Role: LEADER
I20260925 13:11:55.950228  2767 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.18, Last appended by leader: 18, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } }
I20260925 13:11:55.950417  2788 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:55.950498  2788 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:55.950708  2875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:55.950915  2875 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=97af554488bd49f798114e47e4ceba3d]
I20260925 13:11:55.950902  2830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 7 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09"
I20260925 13:11:55.950877  2831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd8c0be10d6644cd822c356e727e40fd" candidate_term: 7 candidate_status { last_received { term: 5 index: 18 } } ignore_live_leader: false dest_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" is_pre_election: true
I20260925 13:11:55.951057  2875 tablet_loader.cc:96] loaded metadata for tablet 06c0fd9861974514a375b7469364c65e (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.951046  2830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 5 FOLLOWER]: Advancing to term 7
I20260925 13:11:55.951107  2875 tablet_loader.cc:96] loaded metadata for tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.951153  2875 tablet_loader.cc:96] loaded metadata for tablet 4035085c2f334a30b294ae2e8184a767 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d])
I20260925 13:11:55.951197  2875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:55.951267  2875 catalog_manager.cc:1259] Loaded cluster ID: 9642985c89344181b175f75e53eae3dc
I20260925 13:11:55.951293  2875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:55.951432  2875 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:55.951519  2875 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd: Loaded TSK: 0
I20260925 13:11:55.951618  2875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:55.951797  2830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8c0be10d6644cd822c356e727e40fd in term 7.
I20260925 13:11:56.060104  2073 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Refusing update from remote peer fd8c0be10d6644cd822c356e727e40fd: Log matching property violated. Preceding OpId in replica: term: 5 index: 18. Preceding OpId from leader: term: 7 index: 19. (index mismatch)
I20260925 13:11:56.060433  2788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260925 13:11:56.060912  2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.061057  2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:56.061367  2767 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.061468  2767 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:56.061566  2830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Refusing update from remote peer fd8c0be10d6644cd822c356e727e40fd: Log matching property violated. Preceding OpId in replica: term: 5 index: 18. Preceding OpId from leader: term: 7 index: 19. (index mismatch)
I20260925 13:11:56.061527  2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.061676  2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:56.061746  2788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260925 13:11:56.062289  2860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c0be10d6644cd822c356e727e40fd. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.062371  2860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:56.062407  2860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.062454  2860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:56.079360  2518 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:56.079403  2439 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:56.079494  2360 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:56.106483   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:56.109863   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.109992   392 tablet_replica.cc:330] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: stopping tablet replica
I20260925 13:11:56.110052   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.110109   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.110318   392 tablet_replica.cc:330] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: stopping tablet replica
I20260925 13:11:56.110354   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.110399   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.110520   392 tablet_replica.cc:330] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: stopping tablet replica
I20260925 13:11:56.110549   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.110595   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.122771   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:56.123958   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.125581  2883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.125587  2882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.125618  2885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.125787   392 server_base.cc:1034] running on GCE node
I20260925 13:11:56.125936   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.125988   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.126006   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916126006 us; error 0 us; skew 500 ppm
I20260925 13:11:56.126818   392 webserver.cc:460] Webserver started at http://127.0.98.1:40585/ using document root <none> and password file <none>
I20260925 13:11:56.126924   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.126959   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:56.127549   392 fs_manager.cc:709] Time spent opening directory manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:56.128103  2890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.128208   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:56.128279   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "919cdc37e0f843f7b2dac0b863a27e22"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:56.128340   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:56.138619   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.138958   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.139137   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:56.139472  2897 ts_tablet_manager.cc:536] Loading tablet metadata (0/3 complete)
I20260925 13:11:56.140751   392 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260925 13:11:56.140818   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:56.140889   392 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20260925 13:11:56.141530  2897 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:56.142326   392 ts_tablet_manager.cc:610] Registered 3 tablets
I20260925 13:11:56.142370   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:56.143429  2897 tablet.cc:1722] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.143497  2897 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.143641  2897 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=151 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.143869  2897 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap complete.
I20260925 13:11:56.144711  2897 ts_tablet_manager.cc:1397] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
I20260925 13:11:56.144858  2897 raft_consensus.cc:348] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.144948  2897 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.145040  2897 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.145256  2897 ts_tablet_manager.cc:1428] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:56.145344  2897 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:56.145970   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:43385
I20260925 13:11:56.146044  2958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:43385 every 8 connection(s)
I20260925 13:11:56.147186  2897 tablet.cc:1722] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.147253  2897 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.147303  2897 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=136 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.147529  2897 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap complete.
I20260925 13:11:56.148357  2897 ts_tablet_manager.cc:1397] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
I20260925 13:11:56.148451  2897 raft_consensus.cc:348] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.148551  2897 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.148613  2897 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.148707  2897 ts_tablet_manager.cc:1428] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.148759  2897 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap starting.
I20260925 13:11:56.150432  2897 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.150492  2897 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.150504  2959 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:56.150556  2897 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.150563  2959 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.150662  2959 heartbeater.cc:511] Master 127.0.98.60:34327 requested a full tablet report, sending...
I20260925 13:11:56.150799  2897 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Bootstrap complete.
I20260925 13:11:56.150990  2819 ts_manager.cc:194] Re-registered known tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:56.152019  2897 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent bootstrapping tablet: real 0.003s	user 0.000s	sys 0.002s
I20260925 13:11:56.152154  2897 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.152238  2897 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.152313  2897 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.152531  2897 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:56.155483  2960 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40169
I20260925 13:11:56.155555  2960 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.155632  2960 heartbeater.cc:511] Master 127.0.98.62:40169 requested a full tablet report, sending...
I20260925 13:11:56.155721  2961 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:56.155759  2961 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.155812  2961 heartbeater.cc:511] Master 127.0.98.61:46353 requested a full tablet report, sending...
I20260925 13:11:56.155916  2061 ts_manager.cc:194] Re-registered known tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:56.156013  2728 ts_manager.cc:194] Re-registered known tserver with Master: 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:56.156742  2728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59780
I20260925 13:11:56.157239  2961 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:56.183471  2788 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "fd8c0be10d6644cd822c356e727e40fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b61637d5b2d427980a42ef58d9f7221" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 40169 } } peers { permanent_uuid: "fd8c0be10d6644cd822c356e727e40fd" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 46353 } } peers { permanent_uuid: "5c1dfdc1200b406fa843ce340fbeaf09" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34327 } } }
I20260925 13:11:56.183633  2788 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:56.346789   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:56.351059   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.351212   392 tablet_replica.cc:330] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: stopping tablet replica
I20260925 13:11:56.351281   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.351353   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.351517   392 tablet_replica.cc:330] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: stopping tablet replica
I20260925 13:11:56.351552   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.351589   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.351673   392 tablet_replica.cc:330] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: stopping tablet replica
I20260925 13:11:56.351701   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.351737   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.364053   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:56.365514   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.367298  2971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.367311  2970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.367368  2973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.367537   392 server_base.cc:1034] running on GCE node
I20260925 13:11:56.367684   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.367710   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.367722   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916367723 us; error 0 us; skew 500 ppm
I20260925 13:11:56.368409   392 webserver.cc:460] Webserver started at http://127.0.98.2:38333/ using document root <none> and password file <none>
I20260925 13:11:56.368525   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.368567   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:56.369311   392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.369998  2978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.370144   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:56.370214   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "a75e127731684428a5e320ef5a358a3e"
format_stamp: "Formatted at 2026-09-25 13:11:53 on dist-test-slave-2538"
I20260925 13:11:56.370282   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260925 13:11:56.395449  2454 consensus_peers.cc:487] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): Couldn't send request to peer a75e127731684428a5e320ef5a358a3e. Status: Network error: Client connection negotiation failed: client connection to 127.0.98.2:40761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.413697   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.413926   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.414111   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:56.415942   392 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260925 13:11:56.416008   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s	user 0.000s	sys 0.000s
I20260925 13:11:56.416494  2986 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:56.416805   392 ts_tablet_manager.cc:610] Registered 3 tablets
I20260925 13:11:56.416869   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:56.418439  2986 tablet.cc:1722] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.418534  2986 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.418601  2986 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=151 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.418900  2986 tablet_bootstrap.cc:492] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Bootstrap complete.
I20260925 13:11:56.419751  2986 ts_tablet_manager.cc:1397] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
I20260925 13:11:56.419914  2986 raft_consensus.cc:348] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.420006  2986 raft_consensus.cc:729] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.420128  2986 consensus_queue.cc:260] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.420323  2986 ts_tablet_manager.cc:1428] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:56.420397  2986 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:56.420684   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:40761
I20260925 13:11:56.420753  3046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:40761 every 8 connection(s)
I20260925 13:11:56.423064  2986 tablet.cc:1722] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.423126  2986 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.423169  2986 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=136 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.423394  2986 tablet_bootstrap.cc:492] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Bootstrap complete.
I20260925 13:11:56.425196  2986 ts_tablet_manager.cc:1397] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260925 13:11:56.425329  2986 raft_consensus.cc:348] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.425410  2986 raft_consensus.cc:729] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.425462  2986 consensus_queue.cc:260] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.425559  2986 ts_tablet_manager.cc:1428] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.425609  2986 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
I20260925 13:11:56.425745  3047 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34327
I20260925 13:11:56.425802  3047 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.425923  3047 heartbeater.cc:511] Master 127.0.98.60:34327 requested a full tablet report, sending...
I20260925 13:11:56.426045  3049 heartbeater.cc:347] Connected to a master server at 127.0.98.61:46353
I20260925 13:11:56.426112  3049 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.426182  3049 heartbeater.cc:511] Master 127.0.98.61:46353 requested a full tablet report, sending...
I20260925 13:11:56.426267  2821 ts_manager.cc:194] Re-registered known tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:56.426461  2728 ts_manager.cc:194] Re-registered known tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:56.426539  2728 catalog_manager.cc:5587] TS a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761) does not have the latest schema for tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d]). Expected version 1 got 0
I20260925 13:11:56.426990  3048 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40169
I20260925 13:11:56.427033  3048 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.427083  3048 heartbeater.cc:511] Master 127.0.98.62:40169 requested a full tablet report, sending...
I20260925 13:11:56.427101  2728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59790
I20260925 13:11:56.427322  2063 ts_manager.cc:194] Re-registered known tserver with Master: a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:56.427587  3049 heartbeater.cc:503] Master 127.0.98.61:46353 was elected leader, sending a full tablet report...
I20260925 13:11:56.427753  2986 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.427805  2986 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.427867  2986 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=1 applied=4 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.427922  2728 catalog_manager.cc:5587] TS a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761) does not have the latest schema for tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d]). Expected version 1 got 0
I20260925 13:11:56.428128  2986 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap complete.
I20260925 13:11:56.429032  2986 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.003s	user 0.002s	sys 0.000s
E20260925 13:11:56.429222  2986 ts_tablet_manager.cc:1439] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Tablet failed to start: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260925 13:11:56.429281  2986 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.429338  2986 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.429416  2986 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.429456  2986 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.455336  2585 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.455464  2585 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.455641  2585 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:56.455860  2931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:56.458986  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:56.459072  3014 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 2.
I20260925 13:11:56.459206  2454 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:56.459336  2585 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:11:56.459378  2585 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.459396  2585 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:56.460219  2585 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.460345  2585 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 election: Requested vote from peers 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385), a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761)
I20260925 13:11:56.460497  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:56.460497  2931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22"
I20260925 13:11:56.460574  3014 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:56.461237  3014 raft_consensus.cc:2454] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2184873e294eb78e5d9ad4f879fe03 in term 3.
I20260925 13:11:56.461436  2454 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e; no voters: 919cdc37e0f843f7b2dac0b863a27e22
I20260925 13:11:56.461690  2585 raft_consensus.cc:2781] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:11:56.461804  2585 raft_consensus.cc:686] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Becoming Leader. State: Replica: 0b2184873e294eb78e5d9ad4f879fe03, State: Running, Role: LEADER
I20260925 13:11:56.461862  2585 consensus_queue.cc:237] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.462538  2728 catalog_manager.cc:5495] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: term changed from 2 to 3, leader changed from a75e127731684428a5e320ef5a358a3e (127.0.98.2) to 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3). New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:56.484097  2454 consensus_queue.cc:579] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: The tablet replica hosted on peer a75e127731684428a5e320ef5a358a3e has failed
I20260925 13:11:56.485148  2489 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.485782  2931 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260925 13:11:56.486105  3065 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:56.487128  3060 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } }
I20260925 13:11:56.487288  2931 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } }
I20260925 13:11:56.488407  2728 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: config changed from index -1 to 5, VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:56.488929  2713 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 077b55ad10884161a8f316758fc55be1 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260925 13:11:56.490317  2489 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
I20260925 13:11:56.490747  2931 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260925 13:11:56.490886  2454 consensus_queue.cc:579] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: The tablet replica hosted on peer a75e127731684428a5e320ef5a358a3e has failed
W20260925 13:11:56.490943  2454 consensus_peers.cc:487] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): Couldn't send request to peer a75e127731684428a5e320ef5a358a3e. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.491034  2585 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:56.492044  3060 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } } }
I20260925 13:11:56.493136  2713 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 077b55ad10884161a8f316758fc55be1 with cas_config_opid_index 5: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260925 13:11:56.493188  2727 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: config changed from index 5 to 6, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } health_report { overall_health: FAILED_UNRECOVERABLE } } }
I20260925 13:11:56.493724  3009 tablet_service.cc:1514] Processing DeleteTablet for tablet 077b55ad10884161a8f316758fc55be1 with delete_type TABLET_DATA_TOMBSTONED (TS a75e127731684428a5e320ef5a358a3e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48696
I20260925 13:11:56.493680  2931 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } } }
I20260925 13:11:56.495321  2489 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.495429  3068 ts_tablet_manager.cc:1905] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260925 13:11:56.495682  2931 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260925 13:11:56.495915  3065 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:11:56.496536  2585 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Committing config change with OpId 3.7: config changed from index 6 to 7, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } }
I20260925 13:11:56.496690  2931 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Committing config change with OpId 3.7: config changed from index 6 to 7, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } }
I20260925 13:11:56.497587  2713 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 077b55ad10884161a8f316758fc55be1 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260925 13:11:56.497733  2727 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: config changed from index 6 to 7, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) evicted. New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:56.499266  2489 consensus_queue.cc:237] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
W20260925 13:11:56.499796  2454 consensus_peers.cc:487] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): Couldn't send request to peer a75e127731684428a5e320ef5a358a3e. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.505700  2948 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.505888  2948 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.506090  2948 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:56.508466  2930 raft_consensus.cc:1261] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20260925 13:11:56.508848  3068 ts_tablet_manager.cc:1918] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260925 13:11:56.508934  3068 log.cc:1196] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/minicluster-data/ts-1-root/wals/077b55ad10884161a8f316758fc55be1
I20260925 13:11:56.509425  2713 catalog_manager.cc:4841] TS a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): tablet 077b55ad10884161a8f316758fc55be1 (table NewTableName [id=97af554488bd49f798114e47e4ceba3d]) successfully deleted
I20260925 13:11:56.509764  3065 consensus_queue.cc:1035] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:11:56.509856  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:56.510002  3014 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 3.
I20260925 13:11:56.510082  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:56.510139  2892 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:56.510340  2948 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 13:11:56.510383  2948 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.510411  2948 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:56.510540  3060 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Committing config change with OpId 3.8: config changed from index 7 to 8, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } } }
I20260925 13:11:56.511178  2948 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.511292  2948 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 4 election: Requested vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:56.511471  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e"
I20260925 13:11:56.511543  3014 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:56.511545  2713 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 077b55ad10884161a8f316758fc55be1 with cas_config_opid_index 7: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260925 13:11:56.511579  2727 catalog_manager.cc:5495] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 reported cstate change: config changed from index 7 to 8, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New cstate: current_term: 3 leader_uuid: "0b2184873e294eb78e5d9ad4f879fe03" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:56.511711  2930 raft_consensus.cc:2932] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Committing config change with OpId 3.8: config changed from index 7 to 8, NON_VOTER a75e127731684428a5e320ef5a358a3e (127.0.98.2) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } } }
I20260925 13:11:56.512434  3014 raft_consensus.cc:2454] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919cdc37e0f843f7b2dac0b863a27e22 in term 4.
I20260925 13:11:56.512641  2892 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e; no voters: 
I20260925 13:11:56.512760  2948 raft_consensus.cc:2781] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 FOLLOWER]: Leader election won for term 4
I20260925 13:11:56.512902  2948 raft_consensus.cc:686] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 LEADER]: Becoming Leader. State: Replica: 919cdc37e0f843f7b2dac0b863a27e22, State: Running, Role: LEADER
I20260925 13:11:56.512949  2948 consensus_queue.cc:237] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.513650  2728 catalog_manager.cc:5495] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 reported cstate change: term changed from 3 to 4, leader changed from a75e127731684428a5e320ef5a358a3e (127.0.98.2) to 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1). New cstate: current_term: 4 leader_uuid: "919cdc37e0f843f7b2dac0b863a27e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:56.513823  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03"
I20260925 13:11:56.529421  2585 raft_consensus.cc:482] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.529531  2585 raft_consensus.cc:504] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.529690  2585 leader_election.cc:288] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385)
I20260925 13:11:56.529844  2930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "919cdc37e0f843f7b2dac0b863a27e22" is_pre_election: true
I20260925 13:11:56.529834  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0fd9861974514a375b7469364c65e" candidate_uuid: "0b2184873e294eb78e5d9ad4f879fe03" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:56.529964  3014 raft_consensus.cc:2379] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b2184873e294eb78e5d9ad4f879fe03 in current term 4: Already voted for candidate 919cdc37e0f843f7b2dac0b863a27e22 in this term.
I20260925 13:11:56.530092  2454 leader_election.cc:302] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b2184873e294eb78e5d9ad4f879fe03; no voters: 919cdc37e0f843f7b2dac0b863a27e22, a75e127731684428a5e320ef5a358a3e
I20260925 13:11:56.530226  2585 raft_consensus.cc:3037] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:11:56.531030  2585 raft_consensus.cc:2726] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:11:56.549474  3069 raft_consensus.cc:482] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a75e127731684428a5e320ef5a358a3e)
I20260925 13:11:56.549594  3069 raft_consensus.cc:504] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } }
I20260925 13:11:56.549819  3069 leader_election.cc:288] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761), 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:56.550076  3014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a75e127731684428a5e320ef5a358a3e" is_pre_election: true
I20260925 13:11:56.550192  3014 raft_consensus.cc:2379] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 919cdc37e0f843f7b2dac0b863a27e22 in current term 3: Already voted for candidate 0b2184873e294eb78e5d9ad4f879fe03 in this term.
I20260925 13:11:56.550139  2489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4035085c2f334a30b294ae2e8184a767" candidate_uuid: "919cdc37e0f843f7b2dac0b863a27e22" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0b2184873e294eb78e5d9ad4f879fe03" is_pre_election: true
I20260925 13:11:56.550376  2892 leader_election.cc:302] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919cdc37e0f843f7b2dac0b863a27e22; no voters: 0b2184873e294eb78e5d9ad4f879fe03, a75e127731684428a5e320ef5a358a3e
I20260925 13:11:56.550503  3069 raft_consensus.cc:3037] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:11:56.551396  3069 raft_consensus.cc:2726] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 13:11:56.551522  2930 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260925 13:11:56.551753  2585 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:56.553948  3014 raft_consensus.cc:1261] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Refusing update from remote peer 0b2184873e294eb78e5d9ad4f879fe03: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260925 13:11:56.554159  3060 consensus_queue.cc:1035] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:56.593917  3014 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260925 13:11:56.594261  3069 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:56.595726  2489 raft_consensus.cc:1261] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Refusing update from remote peer 919cdc37e0f843f7b2dac0b863a27e22: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260925 13:11:56.596077  3069 consensus_queue.cc:1035] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:11:56.618407  3074 ts_tablet_manager.cc:927] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Initiating tablet copy from peer 0b2184873e294eb78e5d9ad4f879fe03 (127.0.98.3:46603)
I20260925 13:11:56.618680  3074 tablet_copy_client.cc:316] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.98.3:46603
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:906: Failure
Failed
Bad status: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260925 13:11:56.621472   392 tablet_server.cc:178] TabletServer@127.0.98.1:43385 shutting down...
I20260925 13:11:56.622201  2500 tablet_copy_service.cc:140] P 0b2184873e294eb78e5d9ad4f879fe03: Received BeginTabletCopySession request for tablet 077b55ad10884161a8f316758fc55be1 from peer a75e127731684428a5e320ef5a358a3e ({username='slave'} at 127.0.0.1:50010)
I20260925 13:11:56.622287  2500 tablet_copy_service.cc:161] P 0b2184873e294eb78e5d9ad4f879fe03: Beginning new tablet copy session on tablet 077b55ad10884161a8f316758fc55be1 from peer a75e127731684428a5e320ef5a358a3e at {username='slave'} at 127.0.0.1:50010: session id = a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1
I20260925 13:11:56.623278  2500 tablet_copy_source_session.cc:205] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:11:56.623942  3074 ts_tablet_manager.cc:1905] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260925 13:11:56.624895   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.625068   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625173   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.625233   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.625543   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625583   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.625607   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P 919cdc37e0f843f7b2dac0b863a27e22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.625839   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.625875   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.625926   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P 919cdc37e0f843f7b2dac0b863a27e22 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.626183  3074 ts_tablet_manager.cc:1918] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 3.4
I20260925 13:11:56.626295  3074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077b55ad10884161a8f316758fc55be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.627189  3074 tablet_copy_client.cc:791] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Starting download of 0 data blocks...
I20260925 13:11:56.627372  3074 tablet_copy_client.cc:655] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Starting download of 1 WAL segments...
I20260925 13:11:56.628625  3074 tablet_copy_client.cc:531] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:11:56.629509  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap starting.
W20260925 13:11:56.630194  2454 consensus_peers.cc:487] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer 919cdc37e0f843f7b2dac0b863a27e22 (127.0.98.1:43385): Couldn't send request to peer 919cdc37e0f843f7b2dac0b863a27e22. Status: Network error: Client connection negotiation failed: client connection to 127.0.98.1:43385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.631425  3074 tablet.cc:1722] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Alter schema from (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    0:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20260925 13:11:56.631477  3074 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260925 13:11:56.631726  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=147 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:56.631919  3074 tablet_bootstrap.cc:492] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Bootstrap complete.
I20260925 13:11:56.632573  3074 ts_tablet_manager.cc:1397] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.001s
I20260925 13:11:56.632726  3074 raft_consensus.cc:348] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
I20260925 13:11:56.632808  3074 raft_consensus.cc:729] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a75e127731684428a5e320ef5a358a3e, State: Initialized, Role: LEARNER
I20260925 13:11:56.632831  3074 consensus_queue.cc:260] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "919cdc37e0f843f7b2dac0b863a27e22" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 43385 } } peers { permanent_uuid: "0b2184873e294eb78e5d9ad4f879fe03" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 46603 } } peers { permanent_uuid: "a75e127731684428a5e320ef5a358a3e" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 40761 } attrs { promote: true } }
I20260925 13:11:56.632975  3074 ts_tablet_manager.cc:1428] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.633312  2500 tablet_copy_service.cc:342] P 0b2184873e294eb78e5d9ad4f879fe03: Request end of tablet copy session a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1 received from {username='slave'} at 127.0.0.1:50010
I20260925 13:11:56.633364  2500 tablet_copy_service.cc:433] P 0b2184873e294eb78e5d9ad4f879fe03: ending tablet copy session a75e127731684428a5e320ef5a358a3e-077b55ad10884161a8f316758fc55be1 on tablet 077b55ad10884161a8f316758fc55be1 with peer a75e127731684428a5e320ef5a358a3e
I20260925 13:11:56.639925   392 tablet_server.cc:194] TabletServer@127.0.98.1:43385 shutdown complete.
I20260925 13:11:56.641166   392 tablet_server.cc:178] TabletServer@127.0.98.2:40761 shutting down...
I20260925 13:11:56.644915   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.645578   392 tablet_replica.cc:330] stopping tablet replica
W20260925 13:11:56.645608  2454 consensus_peers.cc:487] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 -> Peer a75e127731684428a5e320ef5a358a3e (127.0.98.2:40761): Couldn't send request to peer a75e127731684428a5e320ef5a358a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:56.645677   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Raft consensus shutting down.
I20260925 13:11:56.645725   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P a75e127731684428a5e320ef5a358a3e [term 3 LEARNER]: Raft consensus is shut down!
I20260925 13:11:56.646036   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.646080   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.646108   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P a75e127731684428a5e320ef5a358a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.646356   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.646396   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.646451   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P a75e127731684428a5e320ef5a358a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.659016   392 tablet_server.cc:194] TabletServer@127.0.98.2:40761 shutdown complete.
I20260925 13:11:56.660346   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:11:56.663287   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.663407   392 tablet_replica.cc:330] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.663471   392 raft_consensus.cc:2229] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.663578   392 raft_consensus.cc:2258] T 077b55ad10884161a8f316758fc55be1 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.663848   392 tablet_replica.cc:330] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.663892   392 raft_consensus.cc:2229] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.663924   392 raft_consensus.cc:2258] T 06c0fd9861974514a375b7469364c65e P 0b2184873e294eb78e5d9ad4f879fe03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.664131   392 tablet_replica.cc:330] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03: stopping tablet replica
I20260925 13:11:56.664168   392 raft_consensus.cc:2229] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.664218   392 raft_consensus.cc:2258] T 4035085c2f334a30b294ae2e8184a767 P 0b2184873e294eb78e5d9ad4f879fe03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.676481   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:11:56.677732   392 master.cc:537] Master@127.0.98.62:40169 shutting down...
I20260925 13:11:56.680263   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.680326   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.680374   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6b61637d5b2d427980a42ef58d9f7221: stopping tablet replica
W20260925 13:11:56.682209   392 rolling_log.cc:182] Unable to compress old log file: Not found: Unable to open input file to compress: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0/ts_tablet_manager-itest.dist-test-slave-2538.slave.diagnostics.20260925-131152.0.392: No such file or directory (error 2)
I20260925 13:11:56.682346   392 master.cc:552] Master@127.0.98.62:40169 shutdown complete.
I20260925 13:11:56.683591   392 master.cc:537] Master@127.0.98.61:46353 shutting down...
I20260925 13:11:56.686067   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.686133   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd8c0be10d6644cd822c356e727e40fd [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.686151   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.698205   392 master.cc:552] Master@127.0.98.61:46353 shutdown complete.
I20260925 13:11:56.699549   392 master.cc:537] Master@127.0.98.60:34327 shutting down...
I20260925 13:11:56.701961   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.702055   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5c1dfdc1200b406fa843ce340fbeaf09 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.702095   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:56.703748   392 master.cc:552] Master@127.0.98.60:34327 shutdown complete.
I20260925 13:11:56.705485   392 test_util.cc:175] -----------------------------------------------
I20260925 13:11:56.705570   392 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790341876762790-392-0
[  FAILED  ] TsTabletManagerITest.TestTableStats (3725 ms)
[ RUN      ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260925 13:11:56.706635   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:40673
I20260925 13:11:56.706811   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.708397  3085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.708498  3082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.708552  3083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.708765   392 server_base.cc:1034] running on GCE node
I20260925 13:11:56.708858   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.708925   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.708937   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916708937 us; error 0 us; skew 500 ppm
I20260925 13:11:56.709635   392 webserver.cc:460] Webserver started at http://127.0.98.62:36469/ using document root <none> and password file <none>
I20260925 13:11:56.709699   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.709725   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:56.709757   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:56.709993   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "8a088b98920a47ae829e79280159d0b4"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.710868   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.711292  3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.711490   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.711534   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "8a088b98920a47ae829e79280159d0b4"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.711565   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:56.722455   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.722681   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.725741   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:40673
I20260925 13:11:56.728693  3142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:40673 every 8 connection(s)
I20260925 13:11:56.728814  3143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.729900  3143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: Bootstrap starting.
I20260925 13:11:56.730203  3143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:56.730808  3143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: No bootstrap required, opened a new log
I20260925 13:11:56.730957  3143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER }
I20260925 13:11:56.731029  3143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:56.731045  3143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a088b98920a47ae829e79280159d0b4, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.731105  3143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER }
I20260925 13:11:56.731148  3143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:56.731172  3143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:56.731190  3143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:56.731690  3143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER }
I20260925 13:11:56.731748  3143 leader_election.cc:302] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a088b98920a47ae829e79280159d0b4; no voters: 
I20260925 13:11:56.731865  3143 leader_election.cc:288] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:56.731954  3146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:56.732007  3143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:56.732070  3146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 1 LEADER]: Becoming Leader. State: Replica: 8a088b98920a47ae829e79280159d0b4, State: Running, Role: LEADER
I20260925 13:11:56.732123  3146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER }
I20260925 13:11:56.732347  3147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a088b98920a47ae829e79280159d0b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER } }
I20260925 13:11:56.732364  3148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a088b98920a47ae829e79280159d0b4. Latest consensus state: current_term: 1 leader_uuid: "8a088b98920a47ae829e79280159d0b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a088b98920a47ae829e79280159d0b4" member_type: VOTER } }
I20260925 13:11:56.732435  3147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:56.732460  3148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:56.732591  3151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:56.732736  3151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:56.733361  3151 catalog_manager.cc:1347] Generated new cluster ID: 958ede3282994103a20cfeba467ea3d3
I20260925 13:11:56.733358   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:56.733419  3151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:56.746426  3151 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:56.747051  3151 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:56.751444  3151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: Generated new TSK 0
I20260925 13:11:56.751523  3151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:56.765388   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.767407  3165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.767450  3164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.767506   392 server_base.cc:1034] running on GCE node
W20260925 13:11:56.767489  3167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.767756   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.767840   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.767880   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916767880 us; error 0 us; skew 500 ppm
I20260925 13:11:56.768702   392 webserver.cc:460] Webserver started at http://127.0.98.1:36263/ using document root <none> and password file <none>
I20260925 13:11:56.768821   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.768875   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:56.768930   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:56.769325   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "eac60f031d264edab5b16a6933edad19"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.772255   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.772908  3172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.773129   392 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.773198   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "eac60f031d264edab5b16a6933edad19"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.773240   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:56.779742   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.779923   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.780056   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:56.780285   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:56.780319   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.780341   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:56.780354   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.783380   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:39863
I20260925 13:11:56.783414  3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:39863 every 8 connection(s)
I20260925 13:11:56.783792   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.785301  3240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.785279  3243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.785462   392 server_base.cc:1034] running on GCE node
W20260925 13:11:56.785277  3241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.785632   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.785674   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.785688   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916785689 us; error 0 us; skew 500 ppm
I20260925 13:11:56.786419   392 webserver.cc:460] Webserver started at http://127.0.98.2:46491/ using document root <none> and password file <none>
I20260925 13:11:56.786521   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.786568   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:56.786615   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:56.786921   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "755e9afc875d4e42b3674f87ba10aa7e"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.787933   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:56.788571  3236 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40673
I20260925 13:11:56.788576  3248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.788645  3236 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.788724  3236 heartbeater.cc:511] Master 127.0.98.62:40673 requested a full tablet report, sending...
I20260925 13:11:56.788722   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:56.788793   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "755e9afc875d4e42b3674f87ba10aa7e"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.788846   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:56.788985  3107 ts_manager.cc:194] Registered new tserver with Master: eac60f031d264edab5b16a6933edad19 (127.0.98.1:39863)
I20260925 13:11:56.789619  3107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33356
I20260925 13:11:56.804100   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.804324   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.804527   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:56.804749   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:56.804776   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.804826   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:56.804853   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.807955   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:45987
I20260925 13:11:56.808027  3311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:45987 every 8 connection(s)
I20260925 13:11:56.808368   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:56.809773  3316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.809973  3315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:56.809983  3318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:56.810150   392 server_base.cc:1034] running on GCE node
I20260925 13:11:56.810317   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:56.810362   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:56.810375   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341916810376 us; error 0 us; skew 500 ppm
I20260925 13:11:56.811165   392 webserver.cc:460] Webserver started at http://127.0.98.3:42071/ using document root <none> and password file <none>
I20260925 13:11:56.811280   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:56.811336   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:56.811388   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:56.811717   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root/instance:
uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.812316  3312 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40673
I20260925 13:11:56.812389  3312 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.812475  3312 heartbeater.cc:511] Master 127.0.98.62:40673 requested a full tablet report, sending...
I20260925 13:11:56.812718  3107 ts_manager.cc:194] Registered new tserver with Master: 755e9afc875d4e42b3674f87ba10aa7e (127.0.98.2:45987)
I20260925 13:11:56.813001   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:56.813269  3107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33370
I20260925 13:11:56.813635  3323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.813767   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.813838   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root
uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e"
format_stamp: "Formatted at 2026-09-25 13:11:56 on dist-test-slave-2538"
I20260925 13:11:56.813890   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:56.819671   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:56.819943   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:56.820113   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:56.820322   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:56.820369   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.820401   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:56.820427   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.823216   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:41169
I20260925 13:11:56.823324  3386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:41169 every 8 connection(s)
I20260925 13:11:56.826774  3387 heartbeater.cc:347] Connected to a master server at 127.0.98.62:40673
I20260925 13:11:56.826841  3387 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:56.826923  3387 heartbeater.cc:511] Master 127.0.98.62:40673 requested a full tablet report, sending...
I20260925 13:11:56.827205  3107 ts_manager.cc:194] Registered new tserver with Master: 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169)
I20260925 13:11:56.827565  3107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33382
I20260925 13:11:56.827864   392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004507142s
I20260925 13:11:56.836606  3107 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33386:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
  }
}
W20260925 13:11:56.836751  3107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:56.842613  3276 tablet_service.cc:1467] Processing CreateTablet for tablet 040a7c4ce85545f99335068150ae5363 (DEFAULT_TABLE table=test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]), partition=
I20260925 13:11:56.842764  3276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040a7c4ce85545f99335068150ae5363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.843252  3351 tablet_service.cc:1467] Processing CreateTablet for tablet 040a7c4ce85545f99335068150ae5363 (DEFAULT_TABLE table=test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]), partition=
I20260925 13:11:56.843353  3351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040a7c4ce85545f99335068150ae5363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.844312  3403 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Bootstrap starting.
I20260925 13:11:56.844692  3404 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Bootstrap starting.
I20260925 13:11:56.844736  3200 tablet_service.cc:1467] Processing CreateTablet for tablet 040a7c4ce85545f99335068150ae5363 (DEFAULT_TABLE table=test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]), partition=
I20260925 13:11:56.844786  3403 tablet_bootstrap.cc:654] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:56.844877  3200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040a7c4ce85545f99335068150ae5363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.845193  3404 tablet_bootstrap.cc:654] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:56.845451  3403 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: No bootstrap required, opened a new log
I20260925 13:11:56.845542  3403 ts_tablet_manager.cc:1397] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.001s
I20260925 13:11:56.845781  3404 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: No bootstrap required, opened a new log
I20260925 13:11:56.845736  3403 raft_consensus.cc:348] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.845837  3404 ts_tablet_manager.cc:1397] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Time spent bootstrapping tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.845837  3403 raft_consensus.cc:374] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:56.845904  3403 raft_consensus.cc:729] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755e9afc875d4e42b3674f87ba10aa7e, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.845971  3403 consensus_queue.cc:260] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.845990  3404 raft_consensus.cc:348] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.846037  3404 raft_consensus.cc:374] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:56.846069  3404 raft_consensus.cc:729] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903d0c628e3a41fc8f1c6ba1f41fcb4e, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.846148  3403 ts_tablet_manager.cc:1428] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.846127  3404 consensus_queue.cc:260] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.846228  3408 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Bootstrap starting.
I20260925 13:11:56.846292  3404 ts_tablet_manager.cc:1428] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.001s
W20260925 13:11:56.846326  3404 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260925 13:11:56.846364  3387 heartbeater.cc:503] Master 127.0.98.62:40673 was elected leader, sending a full tablet report...
W20260925 13:11:56.846390  3403 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260925 13:11:56.846474  3312 heartbeater.cc:503] Master 127.0.98.62:40673 was elected leader, sending a full tablet report...
I20260925 13:11:56.846788  3408 tablet_bootstrap.cc:654] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:56.847288  3408 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: No bootstrap required, opened a new log
I20260925 13:11:56.847325  3408 ts_tablet_manager.cc:1397] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:56.847430  3408 raft_consensus.cc:348] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.847479  3408 raft_consensus.cc:374] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:56.847492  3408 raft_consensus.cc:729] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac60f031d264edab5b16a6933edad19, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.847553  3408 consensus_queue.cc:260] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.847698  3408 ts_tablet_manager.cc:1428] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:56.847748  3236 heartbeater.cc:503] Master 127.0.98.62:40673 was elected leader, sending a full tablet report...
W20260925 13:11:56.847754  3408 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260925 13:11:56.863299  3407 raft_consensus.cc:482] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:56.863410  3407 raft_consensus.cc:504] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.863659  3407 leader_election.cc:288] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac60f031d264edab5b16a6933edad19 (127.0.98.1:39863), 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169)
I20260925 13:11:56.867656  3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040a7c4ce85545f99335068150ae5363" candidate_uuid: "755e9afc875d4e42b3674f87ba10aa7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac60f031d264edab5b16a6933edad19" is_pre_election: true
I20260925 13:11:56.867801  3210 raft_consensus.cc:2454] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755e9afc875d4e42b3674f87ba10aa7e in term 0.
I20260925 13:11:56.867937  3361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040a7c4ce85545f99335068150ae5363" candidate_uuid: "755e9afc875d4e42b3674f87ba10aa7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" is_pre_election: true
I20260925 13:11:56.867949  3249 leader_election.cc:302] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755e9afc875d4e42b3674f87ba10aa7e, eac60f031d264edab5b16a6933edad19; no voters: 
I20260925 13:11:56.868047  3361 raft_consensus.cc:2454] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755e9afc875d4e42b3674f87ba10aa7e in term 0.
I20260925 13:11:56.868152  3407 raft_consensus.cc:2781] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:56.868184  3407 raft_consensus.cc:482] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:56.868203  3407 raft_consensus.cc:3037] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:56.868875  3407 raft_consensus.cc:504] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.869022  3407 leader_election.cc:288] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [CANDIDATE]: Term 1 election: Requested vote from peers eac60f031d264edab5b16a6933edad19 (127.0.98.1:39863), 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169)
I20260925 13:11:56.869177  3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040a7c4ce85545f99335068150ae5363" candidate_uuid: "755e9afc875d4e42b3674f87ba10aa7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac60f031d264edab5b16a6933edad19"
I20260925 13:11:56.869263  3210 raft_consensus.cc:3037] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:56.869184  3361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040a7c4ce85545f99335068150ae5363" candidate_uuid: "755e9afc875d4e42b3674f87ba10aa7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e"
I20260925 13:11:56.869297  3361 raft_consensus.cc:3037] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:56.869935  3361 raft_consensus.cc:2454] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755e9afc875d4e42b3674f87ba10aa7e in term 1.
I20260925 13:11:56.869941  3210 raft_consensus.cc:2454] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755e9afc875d4e42b3674f87ba10aa7e in term 1.
I20260925 13:11:56.870131  3252 leader_election.cc:302] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755e9afc875d4e42b3674f87ba10aa7e, 903d0c628e3a41fc8f1c6ba1f41fcb4e; no voters: 
I20260925 13:11:56.870273  3407 raft_consensus.cc:2781] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:56.870416  3407 raft_consensus.cc:686] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 LEADER]: Becoming Leader. State: Replica: 755e9afc875d4e42b3674f87ba10aa7e, State: Running, Role: LEADER
I20260925 13:11:56.870458  3407 consensus_queue.cc:237] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.871085  3106 catalog_manager.cc:5495] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 755e9afc875d4e42b3674f87ba10aa7e (127.0.98.2). New cstate: current_term: 1 leader_uuid: "755e9afc875d4e42b3674f87ba10aa7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:56.885443  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:45374
I20260925 13:11:56.896376  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:45374
I20260925 13:11:56.907292  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:45374
I20260925 13:11:56.907521  3416 tablet_replica.cc:330] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: stopping tablet replica
I20260925 13:11:56.907618  3416 raft_consensus.cc:2229] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.907678  3416 raft_consensus.cc:2258] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.908095  3416 ts_tablet_manager.cc:1905] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260925 13:11:56.910118  3416 ts_tablet_manager.cc:1918] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260925 13:11:56.910182  3416 log.cc:1196] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root/wals/040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.911053  3417 ts_tablet_manager.cc:927] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Initiating tablet copy from peer 755e9afc875d4e42b3674f87ba10aa7e (127.0.98.2:45987)
I20260925 13:11:56.911255  3417 tablet_copy_client.cc:316] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.98.2:45987
I20260925 13:11:56.915243  3296 tablet_copy_service.cc:140] P 755e9afc875d4e42b3674f87ba10aa7e: Received BeginTabletCopySession request for tablet 040a7c4ce85545f99335068150ae5363 from peer 903d0c628e3a41fc8f1c6ba1f41fcb4e ({username='slave'} at 127.0.0.1:50386)
I20260925 13:11:56.915315  3296 tablet_copy_service.cc:161] P 755e9afc875d4e42b3674f87ba10aa7e: Beginning new tablet copy session on tablet 040a7c4ce85545f99335068150ae5363 from peer 903d0c628e3a41fc8f1c6ba1f41fcb4e at {username='slave'} at 127.0.0.1:50386: session id = 903d0c628e3a41fc8f1c6ba1f41fcb4e-040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.915817  3296 tablet_copy_source_session.cc:205] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:11:56.916101  3417 ts_tablet_manager.cc:1905] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260925 13:11:56.917718  3417 ts_tablet_manager.cc:1918] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260925 13:11:56.917802  3417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040a7c4ce85545f99335068150ae5363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:56.918345  3417 tablet_copy_client.cc:791] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet copy: Starting download of 0 data blocks...
I20260925 13:11:56.918464  3417 tablet_copy_client.cc:655] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet copy: Starting download of 1 WAL segments...
I20260925 13:11:56.918964  3106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33398:
table { table_id: "cc829ea491ac4af5a67efcb9dbecdcae" table_name: "test-table" }
I20260925 13:11:56.919057  3106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33398:
table { table_id: "cc829ea491ac4af5a67efcb9dbecdcae" table_name: "test-table" }
I20260925 13:11:56.919337  3417 tablet_copy_client.cc:531] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:11:56.919822  3106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: Sending DeleteTablet for 3 replicas of tablet 040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.920125  3200 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:56 UTC) from {username='slave'} at 127.0.0.1:55636
I20260925 13:11:56.920125  3417 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Bootstrap starting.
I20260925 13:11:56.920138  3276 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:56 UTC) from {username='slave'} at 127.0.0.1:50362
I20260925 13:11:56.920425  3419 tablet_replica.cc:330] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: stopping tablet replica
I20260925 13:11:56.920473  3419 raft_consensus.cc:2229] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.920518  3419 raft_consensus.cc:2258] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.920138  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:56 UTC) from {username='slave'} at 127.0.0.1:45346
I20260925 13:11:56.920562  3420 tablet_replica.cc:330] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: stopping tablet replica
I20260925 13:11:56.920603  3420 raft_consensus.cc:2229] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:56.920642  3420 pending_rounds.cc:62] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Trying to abort 1 pending ops.
I20260925 13:11:56.920663  3420 pending_rounds.cc:68] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333240491501907968 op_type: NO_OP noop_request { }
W20260925 13:11:56.920747  3094 catalog_manager.cc:4821] TS 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169): delete failed for tablet 040a7c4ce85545f99335068150ae5363 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 040a7c4ce85545f99335068150ae5363 already in progress: copying tablet
I20260925 13:11:56.920747  3420 raft_consensus.cc:2866] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:11:56.920830  3420 raft_consensus.cc:2258] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.921031  3419 ts_tablet_manager.cc:1905] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:56.921092  3420 ts_tablet_manager.cc:1905] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:56.921859  3417 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260925 13:11:56.922114  3417 tablet_bootstrap.cc:492] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Bootstrap complete.
I20260925 13:11:56.922243  3417 ts_tablet_manager.cc:1397] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.001s
I20260925 13:11:56.922526  3417 raft_consensus.cc:348] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.922554  3419 ts_tablet_manager.cc:1918] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:11:56.922554  3420 ts_tablet_manager.cc:1918] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:11:56.922644  3417 raft_consensus.cc:729] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903d0c628e3a41fc8f1c6ba1f41fcb4e, State: Initialized, Role: FOLLOWER
I20260925 13:11:56.922681  3420 log.cc:1196] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-1-root/wals/040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.922681  3419 log.cc:1196] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-0-root/wals/040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.922755  3417 consensus_queue.cc:260] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755e9afc875d4e42b3674f87ba10aa7e" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 45987 } } peers { permanent_uuid: "eac60f031d264edab5b16a6933edad19" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39863 } } peers { permanent_uuid: "903d0c628e3a41fc8f1c6ba1f41fcb4e" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 41169 } }
I20260925 13:11:56.922904  3419 ts_tablet_manager.cc:1939] T 040a7c4ce85545f99335068150ae5363 P eac60f031d264edab5b16a6933edad19: Deleting consensus metadata
I20260925 13:11:56.922919  3417 ts_tablet_manager.cc:1428] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:56.922904  3420 ts_tablet_manager.cc:1939] T 040a7c4ce85545f99335068150ae5363 P 755e9afc875d4e42b3674f87ba10aa7e: Deleting consensus metadata
W20260925 13:11:56.923002  3417 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260925 13:11:56.923251  3091 catalog_manager.cc:4841] TS 755e9afc875d4e42b3674f87ba10aa7e (127.0.98.2:45987): tablet 040a7c4ce85545f99335068150ae5363 (table test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]) successfully deleted
I20260925 13:11:56.923357  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:56 UTC) from {username='slave'} at 127.0.0.1:45346
I20260925 13:11:56.923525  3091 catalog_manager.cc:4841] TS eac60f031d264edab5b16a6933edad19 (127.0.98.1:39863): tablet 040a7c4ce85545f99335068150ae5363 (table test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]) successfully deleted
W20260925 13:11:56.923641  3094 catalog_manager.cc:4821] TS 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169): delete failed for tablet 040a7c4ce85545f99335068150ae5363 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 040a7c4ce85545f99335068150ae5363 already in progress: copying tablet
I20260925 13:11:56.973738  3296 tablet_copy_service.cc:342] P 755e9afc875d4e42b3674f87ba10aa7e: Request end of tablet copy session 903d0c628e3a41fc8f1c6ba1f41fcb4e-040a7c4ce85545f99335068150ae5363 received from {username='slave'} at 127.0.0.1:50386
I20260925 13:11:56.973934  3296 tablet_copy_service.cc:433] P 755e9afc875d4e42b3674f87ba10aa7e: ending tablet copy session 903d0c628e3a41fc8f1c6ba1f41fcb4e-040a7c4ce85545f99335068150ae5363 on tablet 040a7c4ce85545f99335068150ae5363 with peer 903d0c628e3a41fc8f1c6ba1f41fcb4e
I20260925 13:11:56.973948  3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 040a7c4ce85545f99335068150ae5363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:56 UTC) from {username='slave'} at 127.0.0.1:45346
I20260925 13:11:56.974074  3416 tablet_replica.cc:330] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: stopping tablet replica
I20260925 13:11:56.974136  3416 raft_consensus.cc:2229] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:56.974176  3416 pending_rounds.cc:62] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Trying to abort 1 pending ops.
I20260925 13:11:56.974203  3416 pending_rounds.cc:68] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333240491501907968 op_type: NO_OP noop_request { }
I20260925 13:11:56.974269  3416 raft_consensus.cc:2866] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:11:56.974304  3416 raft_consensus.cc:2258] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:56.974555  3416 ts_tablet_manager.cc:1905] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:56.976190  3416 ts_tablet_manager.cc:1918] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:11:56.976276  3416 log.cc:1196] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790341876762790-392-0/minicluster-data/ts-2-root/wals/040a7c4ce85545f99335068150ae5363
I20260925 13:11:56.976506  3416 ts_tablet_manager.cc:1939] T 040a7c4ce85545f99335068150ae5363 P 903d0c628e3a41fc8f1c6ba1f41fcb4e: Deleting consensus metadata
I20260925 13:11:56.976871  3094 catalog_manager.cc:4841] TS 903d0c628e3a41fc8f1c6ba1f41fcb4e (127.0.98.3:41169): tablet 040a7c4ce85545f99335068150ae5363 (table test-table [id=cc829ea491ac4af5a67efcb9dbecdcae]) successfully deleted
I20260925 13:11:56.985062   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:56.987874   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:56.999941   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:57.000968   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:57.003976   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.015967   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:57.017123   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:11:57.019876   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.032106   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:11:57.033375   392 master.cc:537] Master@127.0.98.62:40673 shutting down...
I20260925 13:11:57.036185   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.036293   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.036413   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8a088b98920a47ae829e79280159d0b4: stopping tablet replica
I20260925 13:11:57.039438   392 master.cc:552] Master@127.0.98.62:40673 shutdown complete.
[       OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (336 ms)
[----------] 4 tests from TsTabletManagerITest (7271 ms total)

[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN      ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260925 13:11:57.043799   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:34183
I20260925 13:11:57.043975   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.045418  3427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.045395  3426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.045560   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.045413  3429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.045719   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.045754   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.045775   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917045776 us; error 0 us; skew 500 ppm
I20260925 13:11:57.046430   392 webserver.cc:460] Webserver started at http://127.0.98.62:45443/ using document root <none> and password file <none>
I20260925 13:11:57.046496   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.046528   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.046561   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.046789   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "6c6d4971d30c4635ad080ffc1521435e"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.047734   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.048236  3434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.048425   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.048486   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "6c6d4971d30c4635ad080ffc1521435e"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.048556   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.056911   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.057216   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.059839   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:34183
I20260925 13:11:57.062119  3486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:34183 every 8 connection(s)
I20260925 13:11:57.062273  3487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.063252  3487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e: Bootstrap starting.
I20260925 13:11:57.063552  3487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.064071  3487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e: No bootstrap required, opened a new log
I20260925 13:11:57.064211  3487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER }
I20260925 13:11:57.064291  3487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.064317  3487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6d4971d30c4635ad080ffc1521435e, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.064404  3487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER }
I20260925 13:11:57.064451  3487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.064481  3487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.064509  3487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.064975  3487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER }
I20260925 13:11:57.065052  3487 leader_election.cc:302] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c6d4971d30c4635ad080ffc1521435e; no voters: 
I20260925 13:11:57.065194  3487 leader_election.cc:288] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.065287  3490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.065407  3487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:57.065433  3490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 1 LEADER]: Becoming Leader. State: Replica: 6c6d4971d30c4635ad080ffc1521435e, State: Running, Role: LEADER
I20260925 13:11:57.065480  3490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER }
I20260925 13:11:57.065675  3492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6d4971d30c4635ad080ffc1521435e. Latest consensus state: current_term: 1 leader_uuid: "6c6d4971d30c4635ad080ffc1521435e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER } }
I20260925 13:11:57.065713  3491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c6d4971d30c4635ad080ffc1521435e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6d4971d30c4635ad080ffc1521435e" member_type: VOTER } }
I20260925 13:11:57.065834  3492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.065851  3491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.066141  3500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:57.066273  3500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:57.066670   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:57.066831  3500 catalog_manager.cc:1347] Generated new cluster ID: dc49a6a435c04941b73d795a213bd5f6
I20260925 13:11:57.066877  3500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:57.072638  3500 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:57.073123  3500 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:57.078488  3500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e: Generated new TSK 0
I20260925 13:11:57.078576  3500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:57.082319   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:57.083917   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.083951  3508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.083998  3511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.084084  3509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.084218   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.084246   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.084259   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917084260 us; error 0 us; skew 500 ppm
I20260925 13:11:57.084971   392 webserver.cc:460] Webserver started at http://127.0.98.1:41889/ using document root <none> and password file <none>
I20260925 13:11:57.085038   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.085137   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.085179   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.085420   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.086364   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:57.086795  3516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.086959   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.087000   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.087060   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.101580   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.101809   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.102224   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:57.102280   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.102321   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:57.102353   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.105801   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:40587
I20260925 13:11:57.106258  3582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:40587 every 8 connection(s)
I20260925 13:11:57.107308  3583 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34183
I20260925 13:11:57.107365  3583 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.107453  3583 heartbeater.cc:511] Master 127.0.98.62:34183 requested a full tablet report, sending...
I20260925 13:11:57.107681  3450 ts_manager.cc:194] Registered new tserver with Master: 9c607ccc9fbc43ecbd90c2a5c2b157df (127.0.98.1:40587)
I20260925 13:11:57.108248   392 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002262714s
I20260925 13:11:57.108433  3450 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59946
I20260925 13:11:57.117784  3547 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260925 13:11:57.117987  3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.119336  3599 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Bootstrap starting.
I20260925 13:11:57.119706  3599 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.120297  3599 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: No bootstrap required, opened a new log
I20260925 13:11:57.120383  3599 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.120518  3599 raft_consensus.cc:348] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.120602  3599 raft_consensus.cc:374] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.120638  3599 raft_consensus.cc:729] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c607ccc9fbc43ecbd90c2a5c2b157df, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.120697  3599 consensus_queue.cc:260] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.120745  3599 raft_consensus.cc:388] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.120771  3599 raft_consensus.cc:482] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.120807  3599 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.121502  3599 raft_consensus.cc:504] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.121598  3599 leader_election.cc:302] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c607ccc9fbc43ecbd90c2a5c2b157df; no voters: 
I20260925 13:11:57.121734  3599 leader_election.cc:288] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.121781  3601 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.121897  3599 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:57.121966  3601 raft_consensus.cc:686] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 LEADER]: Becoming Leader. State: Replica: 9c607ccc9fbc43ecbd90c2a5c2b157df, State: Running, Role: LEADER
I20260925 13:11:57.121969  3583 heartbeater.cc:503] Master 127.0.98.62:34183 was elected leader, sending a full tablet report...
I20260925 13:11:57.122035  3601 consensus_queue.cc:237] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.122270  3603 tablet_replica.cc:439] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: TxnStatusTablet state changed. Reason: New leader 9c607ccc9fbc43ecbd90c2a5c2b157df. Latest consensus state: current_term: 1 leader_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" committed_config { opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } } }
I20260925 13:11:57.122337  3602 tablet_replica.cc:439] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" committed_config { opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } } }
I20260925 13:11:57.122380  3603 tablet_replica.cc:442] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.122413  3602 tablet_replica.cc:442] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: This TxnStatusTablet replica's current role is: LEADER
W20260925 13:11:57.122547  3450 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.122573  3605 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.122786  3605 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:11:57.134975   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:57.137907   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.138200   392 tablet_replica.cc:330] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: stopping tablet replica
I20260925 13:11:57.138273   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.138321   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.150792   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:57.152235   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.153882  3612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.153893  3609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.154217  3610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.157272   392 server_base.cc:1034] running on GCE node
I20260925 13:11:57.157459   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.157508   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.157534   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917157534 us; error 0 us; skew 500 ppm
I20260925 13:11:57.158471   392 webserver.cc:460] Webserver started at http://127.0.98.1:41889/ using document root <none> and password file <none>
I20260925 13:11:57.158579   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.158620   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:57.159310   392 fs_manager.cc:709] Time spent opening directory manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.159895  3617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.160029   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.160080   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.160141   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.176254   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.176489   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.177145  3625 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 13:11:57.178054   392 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:11:57.178109   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:57.178151   392 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 13:11:57.178668   392 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:11:57.178716   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:57.178720  3625 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Bootstrap starting.
I20260925 13:11:57.180280  3625 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:57.180563  3625 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Bootstrap complete.
I20260925 13:11:57.180671  3625 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20260925 13:11:57.180783  3625 raft_consensus.cc:348] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.180855  3625 raft_consensus.cc:729] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c607ccc9fbc43ecbd90c2a5c2b157df, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.180924  3625 consensus_queue.cc:260] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.180974  3625 raft_consensus.cc:388] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.180992  3625 raft_consensus.cc:482] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.181016  3625 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:57.181900  3625 raft_consensus.cc:504] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.181984  3625 leader_election.cc:302] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c607ccc9fbc43ecbd90c2a5c2b157df; no voters: 
I20260925 13:11:57.182089  3625 leader_election.cc:288] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260925 13:11:57.182193  3625 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:57.182265  3680 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:11:57.182354  3680 raft_consensus.cc:686] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 2 LEADER]: Becoming Leader. State: Replica: 9c607ccc9fbc43ecbd90c2a5c2b157df, State: Running, Role: LEADER
I20260925 13:11:57.182390  3680 consensus_queue.cc:237] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } }
I20260925 13:11:57.182538  3680 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" committed_config { opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } } }
I20260925 13:11:57.182578  3680 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.182621   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:40587
I20260925 13:11:57.182672  3680 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 9c607ccc9fbc43ecbd90c2a5c2b157df. Latest consensus state: current_term: 2 leader_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" committed_config { opid_index: -1 peers { permanent_uuid: "9c607ccc9fbc43ecbd90c2a5c2b157df" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40587 } } }
I20260925 13:11:57.182710  3680 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.182992  3693 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.183039  3693 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:11:57.183959  3691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:40587 every 8 connection(s)
I20260925 13:11:57.184579  3694 heartbeater.cc:347] Connected to a master server at 127.0.98.62:34183
I20260925 13:11:57.184636  3694 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.184710  3694 heartbeater.cc:511] Master 127.0.98.62:34183 requested a full tablet report, sending...
I20260925 13:11:57.184948  3450 ts_manager.cc:194] Re-registered known tserver with Master: 9c607ccc9fbc43ecbd90c2a5c2b157df (127.0.98.1:40587)
W20260925 13:11:57.185000  3450 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.185415  3450 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59980
I20260925 13:11:57.185910  3694 heartbeater.cc:503] Master 127.0.98.62:34183 was elected leader, sending a full tablet report...
W20260925 13:11:57.186220  3450 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.311398   392 tablet_server.cc:178] TabletServer@127.0.98.1:40587 shutting down...
I20260925 13:11:57.314229   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.314447   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:57.314517   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 2 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.314575   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 9c607ccc9fbc43ecbd90c2a5c2b157df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.326691   392 tablet_server.cc:194] TabletServer@127.0.98.1:40587 shutdown complete.
I20260925 13:11:57.328941   392 master.cc:537] Master@127.0.98.62:34183 shutting down...
I20260925 13:11:57.332602   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.332672   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.332785   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6c6d4971d30c4635ad080ffc1521435e: stopping tablet replica
I20260925 13:11:57.344969   392 master.cc:552] Master@127.0.98.62:34183 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (305 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260925 13:11:57.348958   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:45935
I20260925 13:11:57.349188   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.350687  3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.350690  3702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.350921  3703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.350947   392 server_base.cc:1034] running on GCE node
I20260925 13:11:57.351145   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.351181   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.351233   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917351232 us; error 0 us; skew 500 ppm
I20260925 13:11:57.352115   392 webserver.cc:460] Webserver started at http://127.0.98.62:34983/ using document root <none> and password file <none>
I20260925 13:11:57.352210   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.352257   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.352305   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.352638   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "2effb710aa0e4935b3861b4e2cf9b36d"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.353760   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:57.354293  3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.354413   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:57.354458   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "2effb710aa0e4935b3861b4e2cf9b36d"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.354511   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.368043   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.368235   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.371044   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:45935
I20260925 13:11:57.373129  3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:45935 every 8 connection(s)
I20260925 13:11:57.373247  3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.374276  3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d: Bootstrap starting.
I20260925 13:11:57.374589  3763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.375174  3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d: No bootstrap required, opened a new log
I20260925 13:11:57.375304  3763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER }
I20260925 13:11:57.375373  3763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.375388  3763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2effb710aa0e4935b3861b4e2cf9b36d, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.375445  3763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER }
I20260925 13:11:57.375504  3763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.375522  3763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.375552  3763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.376030  3763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER }
I20260925 13:11:57.376082  3763 leader_election.cc:302] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2effb710aa0e4935b3861b4e2cf9b36d; no voters: 
I20260925 13:11:57.376206  3763 leader_election.cc:288] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.376288  3766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.376370  3763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:57.376437  3766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 1 LEADER]: Becoming Leader. State: Replica: 2effb710aa0e4935b3861b4e2cf9b36d, State: Running, Role: LEADER
I20260925 13:11:57.376488  3766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER }
I20260925 13:11:57.376705  3767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER } }
I20260925 13:11:57.376726  3768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2effb710aa0e4935b3861b4e2cf9b36d. Latest consensus state: current_term: 1 leader_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2effb710aa0e4935b3861b4e2cf9b36d" member_type: VOTER } }
I20260925 13:11:57.376799  3768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.376978  3773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:57.376789  3767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.377148  3773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:57.377661  3773 catalog_manager.cc:1347] Generated new cluster ID: d6029ec015a048c4b64a4b75d6017196
I20260925 13:11:57.377693   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:57.377708  3773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:57.384119  3773 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:57.384574  3773 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:57.388376  3773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d: Generated new TSK 0
I20260925 13:11:57.388471  3773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:57.393411   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.395128  3785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.395216  3787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.395151  3784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.395290   392 server_base.cc:1034] running on GCE node
I20260925 13:11:57.395473   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.395520   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.395534   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917395534 us; error 0 us; skew 500 ppm
I20260925 13:11:57.396302   392 webserver.cc:460] Webserver started at http://127.0.98.1:43263/ using document root <none> and password file <none>
I20260925 13:11:57.396410   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.396459   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.396507   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.396884   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "d3d93226147446de86444cacd03dc249"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.397914   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.398468  3792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.398643   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.398687   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "d3d93226147446de86444cacd03dc249"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.398749   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.404562   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.404768   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.405200   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:57.405241   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.405263   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:57.405308   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.408874   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:46659
I20260925 13:11:57.408903  3858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:46659 every 8 connection(s)
I20260925 13:11:57.409304   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:57.409314  3859 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45935
I20260925 13:11:57.409420  3859 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.409495  3859 heartbeater.cc:511] Master 127.0.98.62:45935 requested a full tablet report, sending...
I20260925 13:11:57.409708  3727 ts_manager.cc:194] Registered new tserver with Master: d3d93226147446de86444cacd03dc249 (127.0.98.1:46659)
I20260925 13:11:57.410315  3727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50292
W20260925 13:11:57.410815  3869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.410955  3866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.411142   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.411137  3865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.411327   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.411348   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.411360   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917411361 us; error 0 us; skew 500 ppm
I20260925 13:11:57.412353   392 webserver.cc:460] Webserver started at http://127.0.98.2:34067/ using document root <none> and password file <none>
I20260925 13:11:57.412460   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.412528   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.412578   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.412966   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "7072dd600382407bb0f7e82eeaf4e0d3"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.414094   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:57.414659  3874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.414764   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:57.414791   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "7072dd600382407bb0f7e82eeaf4e0d3"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.414832   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.425007   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.425276   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.425661   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:57.425709   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.425751   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:57.425782   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.429342   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:34009
I20260925 13:11:57.429380  3939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:34009 every 8 connection(s)
I20260925 13:11:57.430246  3940 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45935
I20260925 13:11:57.430339  3940 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.430414  3940 heartbeater.cc:511] Master 127.0.98.62:45935 requested a full tablet report, sending...
I20260925 13:11:57.430821  3726 ts_manager.cc:194] Registered new tserver with Master: 7072dd600382407bb0f7e82eeaf4e0d3 (127.0.98.2:34009)
I20260925 13:11:57.431329  3726 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50304
I20260925 13:11:57.431782   392 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.002224764s
I20260925 13:11:57.441226  3823 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260925 13:11:57.441461  3823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.442880  3956 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Bootstrap starting.
I20260925 13:11:57.443337  3956 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.444021  3956 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: No bootstrap required, opened a new log
I20260925 13:11:57.444090  3956 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.444252  3956 raft_consensus.cc:348] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.444330  3956 raft_consensus.cc:374] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.444363  3956 raft_consensus.cc:729] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d93226147446de86444cacd03dc249, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.444414  3956 consensus_queue.cc:260] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.444456  3956 raft_consensus.cc:388] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.444484  3956 raft_consensus.cc:482] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.444517  3956 raft_consensus.cc:3037] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.445063  3956 raft_consensus.cc:504] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.445204  3956 leader_election.cc:302] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3d93226147446de86444cacd03dc249; no voters: 
I20260925 13:11:57.445336  3956 leader_election.cc:288] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.445393  3958 raft_consensus.cc:2781] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.445477  3956 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:57.445545  3958 raft_consensus.cc:686] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 LEADER]: Becoming Leader. State: Replica: d3d93226147446de86444cacd03dc249, State: Running, Role: LEADER
I20260925 13:11:57.445566  3859 heartbeater.cc:503] Master 127.0.98.62:45935 was elected leader, sending a full tablet report...
I20260925 13:11:57.445596  3958 consensus_queue.cc:237] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.445779  3958 tablet_replica.cc:439] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: TxnStatusTablet state changed. Reason: New leader d3d93226147446de86444cacd03dc249. Latest consensus state: current_term: 1 leader_uuid: "d3d93226147446de86444cacd03dc249" committed_config { opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } } }
I20260925 13:11:57.445868  3958 tablet_replica.cc:442] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.446009  3962 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.446074  3962 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:11:57.446061  3959 tablet_replica.cc:439] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3d93226147446de86444cacd03dc249" committed_config { opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } } }
I20260925 13:11:57.446143  3959 tablet_replica.cc:442] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: This TxnStatusTablet replica's current role is: LEADER
W20260925 13:11:57.446202  3726 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260925 13:11:57.446750  3726 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.466295  3967 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Initiating tablet copy from peer d3d93226147446de86444cacd03dc249 (127.0.98.1:46659)
I20260925 13:11:57.466537  3967 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: tablet copy: Beginning tablet copy session from remote peer at address 127.0.98.1:46659
I20260925 13:11:57.469527  3843 tablet_copy_service.cc:140] P d3d93226147446de86444cacd03dc249: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer 7072dd600382407bb0f7e82eeaf4e0d3 ({username='slave'} at 127.0.0.1:54360)
I20260925 13:11:57.469604  3843 tablet_copy_service.cc:161] P d3d93226147446de86444cacd03dc249: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer 7072dd600382407bb0f7e82eeaf4e0d3 at {username='slave'} at 127.0.0.1:54360: session id = 7072dd600382407bb0f7e82eeaf4e0d3-11111111111111111111111111111111
I20260925 13:11:57.470054  3843 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:11:57.470418  3967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.471401  3967 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: tablet copy: Starting download of 0 data blocks...
I20260925 13:11:57.471493  3967 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: tablet copy: Starting download of 1 WAL segments...
I20260925 13:11:57.472131  3967 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:11:57.472725  3967 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Bootstrap starting.
I20260925 13:11:57.474376  3967 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:57.474565  3967 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Bootstrap complete.
I20260925 13:11:57.474681  3967 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20260925 13:11:57.474941  3967 raft_consensus.cc:348] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.475008  3967 raft_consensus.cc:729] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 7072dd600382407bb0f7e82eeaf4e0d3, State: Initialized, Role: NON_PARTICIPANT
I20260925 13:11:57.475032  3967 consensus_queue.cc:260] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.475200  3967 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.475283  3940 heartbeater.cc:503] Master 127.0.98.62:45935 was elected leader, sending a full tablet report...
I20260925 13:11:57.475293  3970 tablet_replica.cc:439] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } } }
I20260925 13:11:57.475452  3970 tablet_replica.cc:442] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260925 13:11:57.475540  3843 tablet_copy_service.cc:342] P d3d93226147446de86444cacd03dc249: Request end of tablet copy session 7072dd600382407bb0f7e82eeaf4e0d3-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:54360
I20260925 13:11:57.475584  3843 tablet_copy_service.cc:433] P d3d93226147446de86444cacd03dc249: ending tablet copy session 7072dd600382407bb0f7e82eeaf4e0d3-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer 7072dd600382407bb0f7e82eeaf4e0d3
W20260925 13:11:57.475656  3726 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.479077   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:57.482137   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.482364   392 tablet_replica.cc:330] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: stopping tablet replica
I20260925 13:11:57.482431   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.482475   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.494977   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:57.496443   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.498116  3974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.498123  3971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.498212   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.498126  3972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.498499   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.498534   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.498565   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917498566 us; error 0 us; skew 500 ppm
I20260925 13:11:57.499145   392 webserver.cc:460] Webserver started at http://127.0.98.1:43263/ using document root <none> and password file <none>
I20260925 13:11:57.499217   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.499238   392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260925 13:11:57.499794   392 fs_manager.cc:709] Time spent opening directory manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.500247  3979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.500427   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.500461   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "d3d93226147446de86444cacd03dc249"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.500505   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.523695   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.523878   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.524986   392 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:11:57.525048   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:57.525489   392 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:11:57.525517  3987 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Bootstrap starting.
I20260925 13:11:57.525530   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.527283  3987 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:57.527582  3987 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Bootstrap complete.
I20260925 13:11:57.527694  3987 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20260925 13:11:57.527832  3987 raft_consensus.cc:348] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.527915  3987 raft_consensus.cc:729] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d93226147446de86444cacd03dc249, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.527971  3987 consensus_queue.cc:260] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.528020  3987 raft_consensus.cc:388] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.528050  3987 raft_consensus.cc:482] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.528072  3987 raft_consensus.cc:3037] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:11:57.528919  3987 raft_consensus.cc:504] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.529014  3987 leader_election.cc:302] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3d93226147446de86444cacd03dc249; no voters: 
I20260925 13:11:57.529145  3987 leader_election.cc:288] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260925 13:11:57.529187   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:46659
I20260925 13:11:57.529292  3987 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20260925 13:11:57.529603  4050 raft_consensus.cc:2781] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:11:57.529767  4050 raft_consensus.cc:686] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 2 LEADER]: Becoming Leader. State: Replica: d3d93226147446de86444cacd03dc249, State: Running, Role: LEADER
I20260925 13:11:57.529824  4050 consensus_queue.cc:237] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } }
I20260925 13:11:57.530021  4050 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader d3d93226147446de86444cacd03dc249. Latest consensus state: current_term: 2 leader_uuid: "d3d93226147446de86444cacd03dc249" committed_config { opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } } }
I20260925 13:11:57.530076  4050 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.530048  4056 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d3d93226147446de86444cacd03dc249" committed_config { opid_index: -1 peers { permanent_uuid: "d3d93226147446de86444cacd03dc249" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46659 } } }
I20260925 13:11:57.530121  4056 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.530205  4051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:46659 every 8 connection(s)
I20260925 13:11:57.530380  4058 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.530418  4058 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:11:57.531335  4054 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45935
I20260925 13:11:57.531401  4054 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.531474  4054 heartbeater.cc:511] Master 127.0.98.62:45935 requested a full tablet report, sending...
I20260925 13:11:57.531819  3726 ts_manager.cc:194] Re-registered known tserver with Master: d3d93226147446de86444cacd03dc249 (127.0.98.1:46659)
W20260925 13:11:57.531889  3726 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.532378  3726 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50318
I20260925 13:11:57.532912  4054 heartbeater.cc:503] Master 127.0.98.62:45935 was elected leader, sending a full tablet report...
W20260925 13:11:57.533146  3726 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:57.658367   392 tablet_server.cc:178] TabletServer@127.0.98.1:46659 shutting down...
I20260925 13:11:57.661525   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.661689   392 tablet_replica.cc:330] stopping tablet replica
I20260925 13:11:57.661763   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 2 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.661821   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P d3d93226147446de86444cacd03dc249 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.673863   392 tablet_server.cc:194] TabletServer@127.0.98.1:46659 shutdown complete.
I20260925 13:11:57.675282   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:11:57.678028   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.678203   392 tablet_replica.cc:330] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3: stopping tablet replica
I20260925 13:11:57.678269   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260925 13:11:57.678329   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 7072dd600382407bb0f7e82eeaf4e0d3 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260925 13:11:57.690274   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:11:57.691753   392 master.cc:537] Master@127.0.98.62:45935 shutting down...
I20260925 13:11:57.694250   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.694340   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.694428   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2effb710aa0e4935b3861b4e2cf9b36d: stopping tablet replica
I20260925 13:11:57.706432   392 master.cc:552] Master@127.0.98.62:45935 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (362 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260925 13:11:57.711108   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:33067
I20260925 13:11:57.711290   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.712785  4064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.713097   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.713227  4067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.713232  4065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.713424   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.713469   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.713481   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917713482 us; error 0 us; skew 500 ppm
I20260925 13:11:57.714208   392 webserver.cc:460] Webserver started at http://127.0.98.62:38355/ using document root <none> and password file <none>
I20260925 13:11:57.714325   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.714377   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.714447   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.714722   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "73598cee5f754048bb33fbb67d6b4e7b"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.715612   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.716089  4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.716213   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.716261   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "73598cee5f754048bb33fbb67d6b4e7b"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.716312   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.721742   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.721903   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.724478   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:33067
I20260925 13:11:57.726210  4124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:33067 every 8 connection(s)
I20260925 13:11:57.726243  4125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.727237  4125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b: Bootstrap starting.
I20260925 13:11:57.727525  4125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.728018  4125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b: No bootstrap required, opened a new log
I20260925 13:11:57.728152  4125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER }
I20260925 13:11:57.728260  4125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.728293  4125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73598cee5f754048bb33fbb67d6b4e7b, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.728350  4125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER }
I20260925 13:11:57.728405  4125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.728426  4125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.728461  4125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.728956  4125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER }
I20260925 13:11:57.729034  4125 leader_election.cc:302] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73598cee5f754048bb33fbb67d6b4e7b; no voters: 
I20260925 13:11:57.729184  4125 leader_election.cc:288] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.729224  4128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.729334  4125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:57.729388  4128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 1 LEADER]: Becoming Leader. State: Replica: 73598cee5f754048bb33fbb67d6b4e7b, State: Running, Role: LEADER
I20260925 13:11:57.729447  4128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER }
I20260925 13:11:57.729646  4128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73598cee5f754048bb33fbb67d6b4e7b. Latest consensus state: current_term: 1 leader_uuid: "73598cee5f754048bb33fbb67d6b4e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER } }
I20260925 13:11:57.729708  4128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.729664  4129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73598cee5f754048bb33fbb67d6b4e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73598cee5f754048bb33fbb67d6b4e7b" member_type: VOTER } }
I20260925 13:11:57.729857  4129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.730082  4134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:57.730223  4134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:57.730523   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:57.730801  4134 catalog_manager.cc:1347] Generated new cluster ID: a740bffe46b049b78bc87678a16ccc2a
I20260925 13:11:57.730895  4134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:57.757275  4134 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:57.757700  4134 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:57.767213  4134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b: Generated new TSK 0
I20260925 13:11:57.767315  4134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:57.794694   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.796444  4147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.796556  4146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.796553  4149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.796900   392 server_base.cc:1034] running on GCE node
I20260925 13:11:57.797001   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.797021   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.797034   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917797034 us; error 0 us; skew 500 ppm
I20260925 13:11:57.797806   392 webserver.cc:460] Webserver started at http://127.0.98.1:36191/ using document root <none> and password file <none>
I20260925 13:11:57.797879   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.797910   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.797945   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.798194   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "a326291a312d4b6d877d9bdd3dfbda32"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.799045   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:57.799496  4154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.799702   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.799741   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "a326291a312d4b6d877d9bdd3dfbda32"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.799768   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.824374   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.824641   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.825029   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:57.825131   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.825172   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:57.825209   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.828925   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:39491
I20260925 13:11:57.828949  4220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:39491 every 8 connection(s)
I20260925 13:11:57.830022  4221 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33067
I20260925 13:11:57.830101  4221 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.830178  4221 heartbeater.cc:511] Master 127.0.98.62:33067 requested a full tablet report, sending...
I20260925 13:11:57.830431  4088 ts_manager.cc:194] Registered new tserver with Master: a326291a312d4b6d877d9bdd3dfbda32 (127.0.98.1:39491)
I20260925 13:11:57.831125  4088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60376
I20260925 13:11:57.831432   392 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00225134s
I20260925 13:11:57.840790  4185 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260925 13:11:57.840953  4185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.842358  4237 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: Bootstrap starting.
I20260925 13:11:57.842693  4237 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.843155  4237 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: No bootstrap required, opened a new log
I20260925 13:11:57.843199  4237 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.843281  4237 raft_consensus.cc:348] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } }
I20260925 13:11:57.843330  4237 raft_consensus.cc:374] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.843343  4237 raft_consensus.cc:729] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a326291a312d4b6d877d9bdd3dfbda32, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.843405  4237 consensus_queue.cc:260] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } }
I20260925 13:11:57.843453  4237 raft_consensus.cc:388] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.843472  4237 raft_consensus.cc:482] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.843495  4237 raft_consensus.cc:3037] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.844010  4237 raft_consensus.cc:504] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } }
I20260925 13:11:57.844066  4237 leader_election.cc:302] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a326291a312d4b6d877d9bdd3dfbda32; no voters: 
I20260925 13:11:57.844177  4237 leader_election.cc:288] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.844213  4239 raft_consensus.cc:2781] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.844316  4237 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.844342  4221 heartbeater.cc:503] Master 127.0.98.62:33067 was elected leader, sending a full tablet report...
I20260925 13:11:57.844331  4239 raft_consensus.cc:686] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 1 LEADER]: Becoming Leader. State: Replica: a326291a312d4b6d877d9bdd3dfbda32, State: Running, Role: LEADER
I20260925 13:11:57.844449  4239 consensus_queue.cc:237] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } }
I20260925 13:11:57.844683  4241 tablet_replica.cc:439] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: TxnStatusTablet state changed. Reason: New leader a326291a312d4b6d877d9bdd3dfbda32. Latest consensus state: current_term: 1 leader_uuid: "a326291a312d4b6d877d9bdd3dfbda32" committed_config { opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } } }
I20260925 13:11:57.844786  4241 tablet_replica.cc:442] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.844751  4240 tablet_replica.cc:439] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a326291a312d4b6d877d9bdd3dfbda32" committed_config { opid_index: -1 peers { permanent_uuid: "a326291a312d4b6d877d9bdd3dfbda32" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 39491 } } }
I20260925 13:11:57.844828  4240 tablet_replica.cc:442] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.844931  4243 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.844981  4243 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260925 13:11:57.844987  4088 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260925 13:11:57.852419  4224 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260925 13:11:57.852540  4224 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260925 13:11:57.852581  4224 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260925 13:11:57.853147  4224 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260925 13:11:57.858616  4224 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260925 13:11:57.858682  4224 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260925 13:11:57.859236  4224 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260925 13:11:57.862339  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.862703  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.862973  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.863271  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.863474  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.863814  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44300
W20260925 13:11:57.866767  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
W20260925 13:11:57.867142  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
W20260925 13:11:57.867419  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
W20260925 13:11:57.867638  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
W20260925 13:11:57.867859  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
W20260925 13:11:57.868173  4185 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44304
I20260925 13:11:57.868769   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:57.871177   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:57.871321   392 tablet_replica.cc:330] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32: stopping tablet replica
I20260925 13:11:57.871368   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.871402   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P a326291a312d4b6d877d9bdd3dfbda32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.883797   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:57.885370   392 master.cc:537] Master@127.0.98.62:33067 shutting down...
I20260925 13:11:57.888398   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:57.888489   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:57.888595   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 73598cee5f754048bb33fbb67d6b4e7b: stopping tablet replica
I20260925 13:11:57.900803   392 master.cc:552] Master@127.0.98.62:33067 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (193 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260925 13:11:57.904765   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:43497
I20260925 13:11:57.904946   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.906400  4253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.906517  4255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.906582   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.906658  4252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.906785   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.906824   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.906850   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917906849 us; error 0 us; skew 500 ppm
I20260925 13:11:57.907618   392 webserver.cc:460] Webserver started at http://127.0.98.62:42285/ using document root <none> and password file <none>
I20260925 13:11:57.907724   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.907770   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.907819   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.908097   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "1f06e966f89b4bddb74a8c8b6624e24f"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.908977   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:57.909490  4260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.909626   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.909688   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "1f06e966f89b4bddb74a8c8b6624e24f"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.909737   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.917732   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.917933   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.920544   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:43497
I20260925 13:11:57.921504  4312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:43497 every 8 connection(s)
I20260925 13:11:57.921598  4313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.922650  4313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f: Bootstrap starting.
I20260925 13:11:57.922942  4313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.923599  4313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f: No bootstrap required, opened a new log
I20260925 13:11:57.923743  4313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER }
I20260925 13:11:57.923799  4313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.923812  4313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f06e966f89b4bddb74a8c8b6624e24f, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.923849  4313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER }
I20260925 13:11:57.923872  4313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.923914  4313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.923942  4313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.924510  4313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER }
I20260925 13:11:57.924567  4313 leader_election.cc:302] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f06e966f89b4bddb74a8c8b6624e24f; no voters: 
I20260925 13:11:57.924698  4313 leader_election.cc:288] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.924824  4316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.924871  4313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:57.925038  4316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 1 LEADER]: Becoming Leader. State: Replica: 1f06e966f89b4bddb74a8c8b6624e24f, State: Running, Role: LEADER
I20260925 13:11:57.925148  4316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER }
I20260925 13:11:57.925346  4317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER } }
I20260925 13:11:57.925431  4317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.925561  4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f06e966f89b4bddb74a8c8b6624e24f. Latest consensus state: current_term: 1 leader_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f06e966f89b4bddb74a8c8b6624e24f" member_type: VOTER } }
I20260925 13:11:57.925616  4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:57.925808  4322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:57.925941  4322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:57.926258   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:57.926486  4322 catalog_manager.cc:1347] Generated new cluster ID: d35bf45e81394df7bd4648e33dd3739a
I20260925 13:11:57.926534  4322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:57.946050  4322 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:57.946533  4322 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:57.953994  4322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f: Generated new TSK 0
I20260925 13:11:57.954103  4322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:57.958149   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:57.959836  4334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.959982   392 server_base.cc:1034] running on GCE node
W20260925 13:11:57.959964  4335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:57.960775  4337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:57.960954   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:57.961018   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:57.961045   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341917961044 us; error 0 us; skew 500 ppm
I20260925 13:11:57.961879   392 webserver.cc:460] Webserver started at http://127.0.98.1:42461/ using document root <none> and password file <none>
I20260925 13:11:57.961982   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:57.962037   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:57.962090   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:57.962472   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "782edff211fc42ae8b25aa5466e7ae97"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.963770   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:57.964452  4342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.964591   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:57.964643   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "782edff211fc42ae8b25aa5466e7ae97"
format_stamp: "Formatted at 2026-09-25 13:11:57 on dist-test-slave-2538"
I20260925 13:11:57.964715   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:57.974571   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:57.974797   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:57.975162   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:57.975194   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.975215   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:57.975251   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:57.978863   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:40359
I20260925 13:11:57.978893  4408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:40359 every 8 connection(s)
I20260925 13:11:57.979581  4409 heartbeater.cc:347] Connected to a master server at 127.0.98.62:43497
I20260925 13:11:57.979651  4409 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:57.979720  4409 heartbeater.cc:511] Master 127.0.98.62:43497 requested a full tablet report, sending...
I20260925 13:11:57.979992  4276 ts_manager.cc:194] Registered new tserver with Master: 782edff211fc42ae8b25aa5466e7ae97 (127.0.98.1:40359)
I20260925 13:11:57.980165   392 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.001109225s
I20260925 13:11:57.980710  4276 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55386
I20260925 13:11:57.988919  4373 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260925 13:11:57.989113  4373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:57.990399  4425 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: Bootstrap starting.
I20260925 13:11:57.990813  4425 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:57.991426  4425 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: No bootstrap required, opened a new log
I20260925 13:11:57.991503  4425 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:57.991673  4425 raft_consensus.cc:348] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:57.991750  4425 raft_consensus.cc:374] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:57.991844  4425 raft_consensus.cc:729] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782edff211fc42ae8b25aa5466e7ae97, State: Initialized, Role: FOLLOWER
I20260925 13:11:57.991914  4425 consensus_queue.cc:260] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:57.992002  4425 raft_consensus.cc:388] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:57.992033  4425 raft_consensus.cc:482] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:57.992062  4425 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:57.992604  4425 raft_consensus.cc:504] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:57.992687  4425 leader_election.cc:302] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782edff211fc42ae8b25aa5466e7ae97; no voters: 
I20260925 13:11:57.992822  4425 leader_election.cc:288] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:57.992875  4427 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:57.992971  4425 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:57.993019  4427 raft_consensus.cc:686] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 LEADER]: Becoming Leader. State: Replica: 782edff211fc42ae8b25aa5466e7ae97, State: Running, Role: LEADER
I20260925 13:11:57.993018  4409 heartbeater.cc:503] Master 127.0.98.62:43497 was elected leader, sending a full tablet report...
I20260925 13:11:57.993115  4427 consensus_queue.cc:237] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:57.993402  4429 tablet_replica.cc:439] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: TxnStatusTablet state changed. Reason: New leader 782edff211fc42ae8b25aa5466e7ae97. Latest consensus state: current_term: 1 leader_uuid: "782edff211fc42ae8b25aa5466e7ae97" committed_config { opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } } }
I20260925 13:11:57.993486  4429 tablet_replica.cc:442] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.993518  4428 tablet_replica.cc:439] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "782edff211fc42ae8b25aa5466e7ae97" committed_config { opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } } }
I20260925 13:11:57.993597  4428 tablet_replica.cc:442] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:11:57.993674  4431 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:11:57.993728  4431 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260925 13:11:57.993850  4276 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260925 13:11:58.008602  4373 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260925 13:11:58.008752  4373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:58.009896  4425 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: Bootstrap starting.
I20260925 13:11:58.010378  4425 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:58.011010  4425 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: No bootstrap required, opened a new log
I20260925 13:11:58.011076  4425 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:58.011241  4425 raft_consensus.cc:348] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:58.011336  4425 raft_consensus.cc:374] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:58.011370  4425 raft_consensus.cc:729] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782edff211fc42ae8b25aa5466e7ae97, State: Initialized, Role: FOLLOWER
I20260925 13:11:58.011422  4425 consensus_queue.cc:260] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:58.011492  4425 raft_consensus.cc:388] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:58.011523  4425 raft_consensus.cc:482] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:58.011554  4425 raft_consensus.cc:3037] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:58.012038  4425 raft_consensus.cc:504] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
I20260925 13:11:58.012120  4425 leader_election.cc:302] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782edff211fc42ae8b25aa5466e7ae97; no voters: 
I20260925 13:11:58.012174  4425 leader_election.cc:288] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:58.012190  4428 raft_consensus.cc:2781] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:58.012238  4428 raft_consensus.cc:686] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 LEADER]: Becoming Leader. State: Replica: 782edff211fc42ae8b25aa5466e7ae97, State: Running, Role: LEADER
I20260925 13:11:58.012259  4425 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20260925 13:11:58.012274  4428 consensus_queue.cc:237] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "782edff211fc42ae8b25aa5466e7ae97" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40359 } }
W20260925 13:11:58.012871  4276 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260925 13:11:58.015537   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:11:58.018235   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:11:58.018390   392 tablet_replica.cc:330] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97: stopping tablet replica
I20260925 13:11:58.018445   392 raft_consensus.cc:2229] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:58.018494   392 raft_consensus.cc:2258] T 22222222222222222222222222222222 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:58.018754   392 tablet_replica.cc:330] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97: stopping tablet replica
I20260925 13:11:58.018791   392 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:58.018815   392 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 782edff211fc42ae8b25aa5466e7ae97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:58.031237   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:11:58.032845   392 master.cc:537] Master@127.0.98.62:43497 shutting down...
I20260925 13:11:58.035354   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:58.035441   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:58.035526   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1f06e966f89b4bddb74a8c8b6624e24f: stopping tablet replica
I20260925 13:11:58.047504   392 master.cc:552] Master@127.0.98.62:43497 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (147 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (1008 ms total)

[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN      ] FailedTabletsAreReplacedITest.OneReplica/0
I20260925 13:11:58.051997   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:43819
I20260925 13:11:58.052184   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:58.053880  4442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.053912  4444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.053939  4441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:58.053941   392 server_base.cc:1034] running on GCE node
I20260925 13:11:58.054205   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:58.054230   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:58.054242   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341918054242 us; error 0 us; skew 500 ppm
I20260925 13:11:58.054910   392 webserver.cc:460] Webserver started at http://127.0.98.62:33055/ using document root <none> and password file <none>
I20260925 13:11:58.054991   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:58.055022   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:58.055092   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:58.055346   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "9544690baefe4d1a943e6d7372c1c451"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.056408   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:58.056959  4449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.057173   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.057230   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "9544690baefe4d1a943e6d7372c1c451"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.057288   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:58.062855   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:58.063050   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:58.065858   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:43819
I20260925 13:11:58.066699  4501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:43819 every 8 connection(s)
I20260925 13:11:58.066800  4502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:58.067759  4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451: Bootstrap starting.
I20260925 13:11:58.068058  4502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:58.068655  4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451: No bootstrap required, opened a new log
I20260925 13:11:58.068814  4502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER }
I20260925 13:11:58.068881  4502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:58.068919  4502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544690baefe4d1a943e6d7372c1c451, State: Initialized, Role: FOLLOWER
I20260925 13:11:58.068990  4502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER }
I20260925 13:11:58.069056  4502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:58.069137  4502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:58.069176  4502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:58.069746  4502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER }
I20260925 13:11:58.069806  4502 leader_election.cc:302] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9544690baefe4d1a943e6d7372c1c451; no voters: 
I20260925 13:11:58.069888  4502 leader_election.cc:288] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:11:58.069921  4505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:58.070060  4505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 1 LEADER]: Becoming Leader. State: Replica: 9544690baefe4d1a943e6d7372c1c451, State: Running, Role: LEADER
I20260925 13:11:58.070063  4502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:58.070116  4505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER }
I20260925 13:11:58.070374  4507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9544690baefe4d1a943e6d7372c1c451. Latest consensus state: current_term: 1 leader_uuid: "9544690baefe4d1a943e6d7372c1c451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER } }
I20260925 13:11:58.070453  4507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:58.070371  4506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9544690baefe4d1a943e6d7372c1c451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9544690baefe4d1a943e6d7372c1c451" member_type: VOTER } }
I20260925 13:11:58.070545  4506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:58.070662  4510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:58.070787  4510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:58.071367   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:11:58.071436  4510 catalog_manager.cc:1347] Generated new cluster ID: ccb1b7c9a71049c5990c6ef074758c50
I20260925 13:11:58.071460  4510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:58.086453  4510 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:58.086966  4510 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:58.090484  4510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451: Generated new TSK 0
I20260925 13:11:58.090574  4510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:58.103338   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:58.105123  4526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.105123  4523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:58.105228   392 server_base.cc:1034] running on GCE node
W20260925 13:11:58.105194  4524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:58.105446   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:58.105484   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:58.105508   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341918105509 us; error 0 us; skew 500 ppm
I20260925 13:11:58.106272   392 webserver.cc:460] Webserver started at http://127.0.98.1:36355/ using document root <none> and password file <none>
I20260925 13:11:58.106345   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:58.106383   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:58.106423   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:58.106845   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "541ee815e43c43779b8510c9934dda83"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.107859   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20260925 13:11:58.108335  4531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.108469   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.108522   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "541ee815e43c43779b8510c9934dda83"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.108559   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:58.116192   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:58.116359   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:58.116480   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:58.116715   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:58.116742   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.116796   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:58.116822   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.119871   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:46647
I20260925 13:11:58.119932  4594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:46647 every 8 connection(s)
I20260925 13:11:58.120268   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:58.121665  4599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.121778  4602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.121826  4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:58.121960   392 server_base.cc:1034] running on GCE node
I20260925 13:11:58.122066   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:58.122112   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:58.122139   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341918122139 us; error 0 us; skew 500 ppm
I20260925 13:11:58.122815   392 webserver.cc:460] Webserver started at http://127.0.98.2:40579/ using document root <none> and password file <none>
I20260925 13:11:58.122923   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:58.122972   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:58.123026   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:58.123311   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "c464ceeebd544f49a7d3ed24227fbb47"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.124580   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260925 13:11:58.125275  4607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.125411  4595 heartbeater.cc:347] Connected to a master server at 127.0.98.62:43819
I20260925 13:11:58.125416   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:58.125484  4595 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:58.125489   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "c464ceeebd544f49a7d3ed24227fbb47"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.125566   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:58.125569  4595 heartbeater.cc:511] Master 127.0.98.62:43819 requested a full tablet report, sending...
I20260925 13:11:58.125869  4466 ts_manager.cc:194] Registered new tserver with Master: 541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647)
I20260925 13:11:58.126454  4466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51390
I20260925 13:11:58.147718   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:58.147993   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:58.148164   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:58.148401   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:58.148442   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.148473   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:58.148492   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.152421   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:42541
I20260925 13:11:58.152510  4670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:42541 every 8 connection(s)
I20260925 13:11:58.152837   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:58.154454  4674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.154556  4677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:58.154623  4675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:58.154855   392 server_base.cc:1034] running on GCE node
I20260925 13:11:58.154982   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:11:58.155019   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:11:58.155033   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341918155033 us; error 0 us; skew 500 ppm
I20260925 13:11:58.155835   392 webserver.cc:460] Webserver started at http://127.0.98.3:42837/ using document root <none> and password file <none>
I20260925 13:11:58.155954   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:58.156031   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:58.156088   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:58.156466   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root/instance:
uuid: "764de6484d2c49e2a1a58e0819448f44"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.157017  4671 heartbeater.cc:347] Connected to a master server at 127.0.98.62:43819
I20260925 13:11:58.157085  4671 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:58.157158  4671 heartbeater.cc:511] Master 127.0.98.62:43819 requested a full tablet report, sending...
I20260925 13:11:58.157378  4466 ts_manager.cc:194] Registered new tserver with Master: c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541)
I20260925 13:11:58.157730   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:11:58.157840  4466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51396
I20260925 13:11:58.158383  4682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.158517   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:11:58.158583   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root
uuid: "764de6484d2c49e2a1a58e0819448f44"
format_stamp: "Formatted at 2026-09-25 13:11:58 on dist-test-slave-2538"
I20260925 13:11:58.158629   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:58.175390   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:11:58.175661   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:58.175863   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:58.176100   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:58.176152   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.176198   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:58.176227   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.179917   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:44943
I20260925 13:11:58.180004  4745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:44943 every 8 connection(s)
I20260925 13:11:58.183729  4746 heartbeater.cc:347] Connected to a master server at 127.0.98.62:43819
I20260925 13:11:58.183830  4746 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:58.183887  4746 heartbeater.cc:511] Master 127.0.98.62:43819 requested a full tablet report, sending...
I20260925 13:11:58.184094  4466 ts_manager.cc:194] Registered new tserver with Master: 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943)
I20260925 13:11:58.184507  4466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51408
I20260925 13:11:58.184540   392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004431487s
I20260925 13:11:58.190244   392 test_util.cc:268] Using random seed: 1980721826
I20260925 13:11:58.195791  4466 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51432:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260925 13:11:58.201663  4559 tablet_service.cc:1467] Processing CreateTablet for tablet c9611e0a82204cb5aa45980576fa2549 (DEFAULT_TABLE table=test-workload [id=c0977523eee94d1b92b66dab6121367e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:58.201828  4559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9611e0a82204cb5aa45980576fa2549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:58.201936  4635 tablet_service.cc:1467] Processing CreateTablet for tablet c9611e0a82204cb5aa45980576fa2549 (DEFAULT_TABLE table=test-workload [id=c0977523eee94d1b92b66dab6121367e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:58.202025  4635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9611e0a82204cb5aa45980576fa2549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:58.203445  4767 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Bootstrap starting.
I20260925 13:11:58.203613  4710 tablet_service.cc:1467] Processing CreateTablet for tablet c9611e0a82204cb5aa45980576fa2549 (DEFAULT_TABLE table=test-workload [id=c0977523eee94d1b92b66dab6121367e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:58.203732  4710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9611e0a82204cb5aa45980576fa2549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:58.203920  4767 tablet_bootstrap.cc:654] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:58.204327  4766 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: Bootstrap starting.
I20260925 13:11:58.204588  4767 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: No bootstrap required, opened a new log
I20260925 13:11:58.204658  4767 ts_tablet_manager.cc:1397] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:58.204859  4766 tablet_bootstrap.cc:654] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:58.204852  4767 raft_consensus.cc:348] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.204965  4767 raft_consensus.cc:374] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:58.204993  4767 raft_consensus.cc:729] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 541ee815e43c43779b8510c9934dda83, State: Initialized, Role: FOLLOWER
I20260925 13:11:58.205053  4767 consensus_queue.cc:260] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.205252  4767 ts_tablet_manager.cc:1428] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:58.205343  4595 heartbeater.cc:503] Master 127.0.98.62:43819 was elected leader, sending a full tablet report...
I20260925 13:11:58.205374  4769 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Bootstrap starting.
I20260925 13:11:58.205543  4766 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: No bootstrap required, opened a new log
I20260925 13:11:58.205720  4766 ts_tablet_manager.cc:1397] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: Time spent bootstrapping tablet: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:11:58.205812  4769 tablet_bootstrap.cc:654] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:58.205982  4766 raft_consensus.cc:348] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.206075  4766 raft_consensus.cc:374] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:58.206104  4766 raft_consensus.cc:729] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c464ceeebd544f49a7d3ed24227fbb47, State: Initialized, Role: FOLLOWER
I20260925 13:11:58.206163  4766 consensus_queue.cc:260] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.206324  4766 ts_tablet_manager.cc:1428] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:11:58.206389  4671 heartbeater.cc:503] Master 127.0.98.62:43819 was elected leader, sending a full tablet report...
I20260925 13:11:58.206611  4769 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: No bootstrap required, opened a new log
I20260925 13:11:58.206646  4769 ts_tablet_manager.cc:1397] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:58.206780  4769 raft_consensus.cc:348] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.206835  4769 raft_consensus.cc:374] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:58.206874  4769 raft_consensus.cc:729] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 764de6484d2c49e2a1a58e0819448f44, State: Initialized, Role: FOLLOWER
I20260925 13:11:58.206913  4769 consensus_queue.cc:260] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.207068  4769 ts_tablet_manager.cc:1428] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:58.207119  4746 heartbeater.cc:503] Master 127.0.98.62:43819 was elected leader, sending a full tablet report...
I20260925 13:11:58.211476  4770 raft_consensus.cc:482] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:58.211539  4770 raft_consensus.cc:504] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.211689  4770 leader_election.cc:288] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943), c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541)
I20260925 13:11:58.214886  4645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9611e0a82204cb5aa45980576fa2549" candidate_uuid: "541ee815e43c43779b8510c9934dda83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c464ceeebd544f49a7d3ed24227fbb47" is_pre_election: true
I20260925 13:11:58.214886  4720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9611e0a82204cb5aa45980576fa2549" candidate_uuid: "541ee815e43c43779b8510c9934dda83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "764de6484d2c49e2a1a58e0819448f44" is_pre_election: true
I20260925 13:11:58.215029  4645 raft_consensus.cc:2454] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 541ee815e43c43779b8510c9934dda83 in term 0.
I20260925 13:11:58.215032  4720 raft_consensus.cc:2454] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 541ee815e43c43779b8510c9934dda83 in term 0.
I20260925 13:11:58.215179  4533 leader_election.cc:302] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 541ee815e43c43779b8510c9934dda83, 764de6484d2c49e2a1a58e0819448f44; no voters: 
I20260925 13:11:58.215320  4770 raft_consensus.cc:2781] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:58.215390  4770 raft_consensus.cc:482] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:58.215422  4770 raft_consensus.cc:3037] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:58.216023  4770 raft_consensus.cc:504] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.216166  4770 leader_election.cc:288] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [CANDIDATE]: Term 1 election: Requested vote from peers 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943), c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541)
I20260925 13:11:58.216293  4645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9611e0a82204cb5aa45980576fa2549" candidate_uuid: "541ee815e43c43779b8510c9934dda83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c464ceeebd544f49a7d3ed24227fbb47"
I20260925 13:11:58.216361  4645 raft_consensus.cc:3037] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:58.216297  4720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9611e0a82204cb5aa45980576fa2549" candidate_uuid: "541ee815e43c43779b8510c9934dda83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "764de6484d2c49e2a1a58e0819448f44"
I20260925 13:11:58.216418  4720 raft_consensus.cc:3037] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:58.216960  4645 raft_consensus.cc:2454] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 541ee815e43c43779b8510c9934dda83 in term 1.
I20260925 13:11:58.217124  4535 leader_election.cc:302] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 541ee815e43c43779b8510c9934dda83, c464ceeebd544f49a7d3ed24227fbb47; no voters: 
I20260925 13:11:58.217235  4770 raft_consensus.cc:2781] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:58.217254  4720 raft_consensus.cc:2454] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 541ee815e43c43779b8510c9934dda83 in term 1.
I20260925 13:11:58.217403  4770 raft_consensus.cc:686] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Becoming Leader. State: Replica: 541ee815e43c43779b8510c9934dda83, State: Running, Role: LEADER
I20260925 13:11:58.217465  4770 consensus_queue.cc:237] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:58.217952  4466 catalog_manager.cc:5495] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 541ee815e43c43779b8510c9934dda83 (127.0.98.1). New cstate: current_term: 1 leader_uuid: "541ee815e43c43779b8510c9934dda83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:58.248364  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:58.248378  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:58.248580  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
All reported replicas are:
  A = 764de6484d2c49e2a1a58e0819448f44
  B = 541ee815e43c43779b8510c9934dda83
  C = c464ceeebd544f49a7d3ed24227fbb47
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A   B*  C    |              |              | Yes
 A             | A   B   C    | 1            | -1           | Yes
 B             | A   B*  C    | 1            | -1           | Yes
 C             | A   B   C    | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:58.258271   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:58.434595  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:58.434934  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:58.434943  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
All reported replicas are:
  A = 764de6484d2c49e2a1a58e0819448f44
  B = 541ee815e43c43779b8510c9934dda83
  C = c464ceeebd544f49a7d3ed24227fbb47
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A   B*  C    |              |              | Yes
 A             | A   B   C    | 1            | -1           | Yes
 B             | A   B*  C    | 1            | -1           | Yes
 C             | A   B   C    | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:58.443547   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:58.695349  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:58.695590  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:58.695675  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:58.697111  4770 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:58.700368  4776 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
All reported replicas are:
  A = 764de6484d2c49e2a1a58e0819448f44
  B = 541ee815e43c43779b8510c9934dda83
  C = c464ceeebd544f49a7d3ed24227fbb47
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A   B*  C    |              |              | Yes
 A             | A   B   C    | 1            | -1           | Yes
 B             | A   B*  C    | 1            | -1           | Yes
 C             | A   B   C    | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:58.704321   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:59.069478  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.069491  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.070154  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260925 13:11:59.077850   392 log_verifier.cc:126] Checking tablet c9611e0a82204cb5aa45980576fa2549
I20260925 13:11:59.095908   392 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c9611e0a82204cb5aa45980576fa2549
I20260925 13:11:59.096031   392 tablet_replica.cc:330] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: stopping tablet replica
I20260925 13:11:59.096093   392 raft_consensus.cc:2229] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:59.096143   392 raft_consensus.cc:2258] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:11:59.119859  4533 consensus_peers.cc:487] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 -> Peer 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): Couldn't send request to peer 764de6484d2c49e2a1a58e0819448f44. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:59.119907  4776 raft_consensus.cc:998] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Attempting to remove follower 764de6484d2c49e2a1a58e0819448f44 from the Raft config. Reason: The tablet replica hosted on peer 764de6484d2c49e2a1a58e0819448f44 has failed
I20260925 13:11:59.120050  4776 consensus_queue.cc:237] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:59.120522  4645 raft_consensus.cc:1261] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Refusing update from remote peer 541ee815e43c43779b8510c9934dda83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:59.120793  4770 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:11:59.121976  4770 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.123746  4464 catalog_manager.cc:5495] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 reported cstate change: config changed from index -1 to 2, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New cstate: current_term: 1 leader_uuid: "541ee815e43c43779b8510c9934dda83" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.124132  4645 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.125310  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:59.126122  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.126487  4710 tablet_service.cc:1514] Processing DeleteTablet for tablet c9611e0a82204cb5aa45980576fa2549 with delete_type TABLET_DATA_TOMBSTONED (TS 764de6484d2c49e2a1a58e0819448f44 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:47368
I20260925 13:11:59.126497  4567 consensus_queue.cc:237] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } }
I20260925 13:11:59.126973  4645 raft_consensus.cc:1261] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Refusing update from remote peer 541ee815e43c43779b8510c9934dda83: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260925 13:11:59.127023  4533 consensus_peers.cc:487] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 -> Peer 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): Couldn't send request to peer 764de6484d2c49e2a1a58e0819448f44. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:59.127106  4770 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 13:11:59.127626  4770 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.128634  4452 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet c9611e0a82204cb5aa45980576fa2549 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260925 13:11:59.128753  4464 catalog_manager.cc:5495] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 reported cstate change: config changed from index 2 to 3, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New cstate: current_term: 1 leader_uuid: "541ee815e43c43779b8510c9934dda83" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.129137  4645 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.129675  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.131172  4906 ts_tablet_manager.cc:1905] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260925 13:11:59.132395  4906 ts_tablet_manager.cc:1918] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260925 13:11:59.132481  4906 log.cc:1196] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790341876762790-392-0/minicluster-data/ts-2-root/wals/c9611e0a82204cb5aa45980576fa2549
I20260925 13:11:59.132809  4451 catalog_manager.cc:4841] TS 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): tablet c9611e0a82204cb5aa45980576fa2549 (table test-workload [id=c0977523eee94d1b92b66dab6121367e]) successfully deleted
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): not running
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
All reported replicas are:
  A = 764de6484d2c49e2a1a58e0819448f44
  B = 541ee815e43c43779b8510c9934dda83
  C = c464ceeebd544f49a7d3ed24227fbb47
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B*  C              |              |              | Yes
 A             | [config not available] |              |              | 
 B             |     B*  C              | 1            | 2            | Yes
 C             | A   B*  C              | 1            | 3            | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:59.136449   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:59.313185  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.313446  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.314311  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): not running
    State:       FAILED
    Data state:  TABLET_DATA_TOMBSTONED
    Last status: Deleted tablet blocks from disk
All reported replicas are:
  A = 541ee815e43c43779b8510c9934dda83
  B = c464ceeebd544f49a7d3ed24227fbb47
  C = 764de6484d2c49e2a1a58e0819448f44
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | A*  B   C              | 1            | 3            | Yes
 B             | A*  B   C              | 1            | 3            | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:59.321899   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:59.554517  4776 raft_consensus.cc:998] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Attempting to remove follower 764de6484d2c49e2a1a58e0819448f44 from the Raft config. Reason: The tablet replica hosted on peer 764de6484d2c49e2a1a58e0819448f44 has failed
I20260925 13:11:59.554678  4776 consensus_queue.cc:237] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } }
I20260925 13:11:59.555074  4644 raft_consensus.cc:1261] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Refusing update from remote peer 541ee815e43c43779b8510c9934dda83: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260925 13:11:59.555317  4776 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260925 13:11:59.556000  4770 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.556160  4644 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.557260  4465 catalog_manager.cc:5495] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 reported cstate change: config changed from index 3 to 4, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) evicted. New cstate: current_term: 1 leader_uuid: "541ee815e43c43779b8510c9934dda83" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } }
I20260925 13:11:59.558023  4710 tablet_service.cc:1514] Processing DeleteTablet for tablet c9611e0a82204cb5aa45980576fa2549 with delete_type TABLET_DATA_TOMBSTONED (TS 764de6484d2c49e2a1a58e0819448f44 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47368
I20260925 13:11:59.558219  4567 consensus_queue.cc:237] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } }
I20260925 13:11:59.558637  4644 raft_consensus.cc:1261] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Refusing update from remote peer 541ee815e43c43779b8510c9934dda83: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260925 13:11:59.558691  4533 consensus_peers.cc:487] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 -> Peer 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): Couldn't send request to peer 764de6484d2c49e2a1a58e0819448f44. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:11:59.558873  4770 consensus_queue.cc:1035] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 13:11:59.559091  4451 catalog_manager.cc:4841] TS 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): tablet c9611e0a82204cb5aa45980576fa2549 (table test-workload [id=c0977523eee94d1b92b66dab6121367e]) successfully deleted
I20260925 13:11:59.559335  4770 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.559510  4644 raft_consensus.cc:2932] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.560298  4452 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet c9611e0a82204cb5aa45980576fa2549 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260925 13:11:59.560405  4465 catalog_manager.cc:5495] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 reported cstate change: config changed from index 4 to 5, VOTER 764de6484d2c49e2a1a58e0819448f44 (127.0.98.3) added. New cstate: current_term: 1 leader_uuid: "541ee815e43c43779b8510c9934dda83" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } } }
I20260925 13:11:59.575443  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.575544  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:59.576102  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): not running
    State:       FAILED
    Data state:  TABLET_DATA_TOMBSTONED
    Last status: Deleted tablet blocks from disk
All reported replicas are:
  A = 541ee815e43c43779b8510c9934dda83
  B = c464ceeebd544f49a7d3ed24227fbb47
  C = 764de6484d2c49e2a1a58e0819448f44
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | A*  B   C              | 1            | 5            | Yes
 B             | A*  B   C              | 1            | 5            | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:59.583552   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:59.948482  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:11:59.948536  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.948786  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:11:59.954367  4977 ts_tablet_manager.cc:927] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Initiating tablet copy from peer 541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647)
I20260925 13:11:59.954902  4977 tablet_copy_client.cc:316] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet copy: Beginning tablet copy session from remote peer at address 127.0.98.1:46647
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): not running
    State:       FAILED
    Data state:  TABLET_DATA_TOMBSTONED
    Last status: Deleted tablet blocks from disk
All reported replicas are:
  A = 541ee815e43c43779b8510c9934dda83
  B = c464ceeebd544f49a7d3ed24227fbb47
  C = 764de6484d2c49e2a1a58e0819448f44
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | A*  B   C              | 1            | 5            | Yes
 B             | A*  B   C              | 1            | 5            | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:11:59.957479   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:11:59.958487  4579 tablet_copy_service.cc:140] P 541ee815e43c43779b8510c9934dda83: Received BeginTabletCopySession request for tablet c9611e0a82204cb5aa45980576fa2549 from peer 764de6484d2c49e2a1a58e0819448f44 ({username='slave'} at 127.0.0.1:35588)
I20260925 13:11:59.958560  4579 tablet_copy_service.cc:161] P 541ee815e43c43779b8510c9934dda83: Beginning new tablet copy session on tablet c9611e0a82204cb5aa45980576fa2549 from peer 764de6484d2c49e2a1a58e0819448f44 at {username='slave'} at 127.0.0.1:35588: session id = 764de6484d2c49e2a1a58e0819448f44-c9611e0a82204cb5aa45980576fa2549
I20260925 13:11:59.959040  4579 tablet_copy_source_session.cc:205] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:11:59.959352  4977 ts_tablet_manager.cc:1905] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260925 13:11:59.960971  4977 ts_tablet_manager.cc:1918] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260925 13:11:59.961046  4977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9611e0a82204cb5aa45980576fa2549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:59.961658  4977 tablet_copy_client.cc:791] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet copy: Starting download of 0 data blocks...
I20260925 13:11:59.961777  4977 tablet_copy_client.cc:655] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet copy: Starting download of 1 WAL segments...
I20260925 13:11:59.962466  4977 tablet_copy_client.cc:531] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:11:59.962970  4977 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Bootstrap starting.
I20260925 13:11:59.964223  4977 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:11:59.964362  4977 tablet_bootstrap.cc:492] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Bootstrap complete.
I20260925 13:11:59.964426  4977 ts_tablet_manager.cc:1397] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:11:59.964526  4977 raft_consensus.cc:348] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } }
I20260925 13:11:59.964632  4977 raft_consensus.cc:729] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 764de6484d2c49e2a1a58e0819448f44, State: Initialized, Role: FOLLOWER
I20260925 13:11:59.964679  4977 consensus_queue.cc:260] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "541ee815e43c43779b8510c9934dda83" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 46647 } } peers { permanent_uuid: "c464ceeebd544f49a7d3ed24227fbb47" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 42541 } } peers { permanent_uuid: "764de6484d2c49e2a1a58e0819448f44" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 44943 } }
I20260925 13:11:59.964820  4977 ts_tablet_manager.cc:1428] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: Time spent starting tablet: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:11:59.965025  4579 tablet_copy_service.cc:342] P 541ee815e43c43779b8510c9934dda83: Request end of tablet copy session 764de6484d2c49e2a1a58e0819448f44-c9611e0a82204cb5aa45980576fa2549 received from {username='slave'} at 127.0.0.1:35588
I20260925 13:11:59.965103  4579 tablet_copy_service.cc:433] P 541ee815e43c43779b8510c9934dda83: ending tablet copy session 764de6484d2c49e2a1a58e0819448f44-c9611e0a82204cb5aa45980576fa2549 on tablet c9611e0a82204cb5aa45980576fa2549 with peer 764de6484d2c49e2a1a58e0819448f44
I20260925 13:12:00.497947  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:00.498140  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:00.498289  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet c9611e0a82204cb5aa45980576fa2549 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
  541ee815e43c43779b8510c9934dda83 (127.0.98.1:46647): RUNNING [LEADER]
  c464ceeebd544f49a7d3ed24227fbb47 (127.0.98.2:42541): RUNNING
  764de6484d2c49e2a1a58e0819448f44 (127.0.98.3:44943): RUNNING
All reported replicas are:
  A = 541ee815e43c43779b8510c9934dda83
  B = c464ceeebd544f49a7d3ed24227fbb47
  C = 764de6484d2c49e2a1a58e0819448f44
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C    |              |              | Yes
 A             | A*  B   C    | 1            | 5            | Yes
 B             | A*  B   C    | 1            | 5            | Yes
 C             | A   B   C    | 1            | 5            | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:00.507398   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:00.573770  4720 raft_consensus.cc:1203] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5]   Dedup: 1.5->[]
I20260925 13:12:01.294255  4559 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:01.294724  4710 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.294746  4635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 9544690baefe4d1a943e6d7372c1c451 | 127.0.98.62:43819 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 541ee815e43c43779b8510c9934dda83 | 127.0.98.1:46647 | HEALTHY | <none>   |       1        |       0
 764de6484d2c49e2a1a58e0819448f44 | 127.0.98.3:44943 | HEALTHY | <none>   |       0        |       0
 c464ceeebd544f49a7d3ed24227fbb47 | 127.0.98.2:42541 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260925 13:12:01.304977   392 log_verifier.cc:126] Checking tablet c9611e0a82204cb5aa45980576fa2549
I20260925 13:12:01.316708   392 log_verifier.cc:177] Verified matching terms for 5 ops in tablet c9611e0a82204cb5aa45980576fa2549
I20260925 13:12:01.317906   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:12:01.321103   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:01.321228   392 tablet_replica.cc:330] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83: stopping tablet replica
I20260925 13:12:01.321287   392 raft_consensus.cc:2229] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:12:01.321362   392 raft_consensus.cc:2258] T c9611e0a82204cb5aa45980576fa2549 P 541ee815e43c43779b8510c9934dda83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:01.334211   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:12:01.335261   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:12:01.338732   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:01.338889   392 tablet_replica.cc:330] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47: stopping tablet replica
I20260925 13:12:01.338932   392 raft_consensus.cc:2229] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:12:01.339013   392 raft_consensus.cc:2258] T c9611e0a82204cb5aa45980576fa2549 P c464ceeebd544f49a7d3ed24227fbb47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:01.351706   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:12:01.352757   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:12:01.355816   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:01.355958   392 tablet_replica.cc:330] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44: stopping tablet replica
I20260925 13:12:01.356014   392 raft_consensus.cc:2229] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:12:01.356063   392 raft_consensus.cc:2258] T c9611e0a82204cb5aa45980576fa2549 P 764de6484d2c49e2a1a58e0819448f44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:01.368691   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:12:01.369805   392 master.cc:537] Master@127.0.98.62:43819 shutting down...
I20260925 13:12:01.372691   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:12:01.372771   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:01.372823   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9544690baefe4d1a943e6d7372c1c451: stopping tablet replica
I20260925 13:12:01.384637   392 master.cc:552] Master@127.0.98.62:43819 shutdown complete.
[       OK ] FailedTabletsAreReplacedITest.OneReplica/0 (3337 ms)
[ RUN      ] FailedTabletsAreReplacedITest.OneReplica/1
I20260925 13:12:01.389416   392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:41763
I20260925 13:12:01.389596   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:12:01.391113  5041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.391121  5044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.391108   392 server_base.cc:1034] running on GCE node
W20260925 13:12:01.391110  5042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.391510   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:12:01.391566   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:12:01.391595   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341921391595 us; error 0 us; skew 500 ppm
I20260925 13:12:01.392232   392 webserver.cc:460] Webserver started at http://127.0.98.62:34687/ using document root <none> and password file <none>
I20260925 13:12:01.392333   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:12:01.392379   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:12:01.392424   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:12:01.392717   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/master-0-root/instance:
uuid: "c232b126b78f4f9b8169b106478c3323"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.393754   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:12:01.394232  5049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.394505   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.394558   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/master-0-root
uuid: "c232b126b78f4f9b8169b106478c3323"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.394611   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:12:01.405902   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:12:01.406116   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:12:01.408713   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:41763
I20260925 13:12:01.411208  5101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:41763 every 8 connection(s)
I20260925 13:12:01.411321  5102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:12:01.412371  5102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323: Bootstrap starting.
I20260925 13:12:01.412688  5102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323: Neither blocks nor log segments found. Creating new log.
I20260925 13:12:01.413327  5102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323: No bootstrap required, opened a new log
I20260925 13:12:01.413492  5102 raft_consensus.cc:348] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER }
I20260925 13:12:01.413565  5102 raft_consensus.cc:374] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:12:01.413589  5102 raft_consensus.cc:729] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c232b126b78f4f9b8169b106478c3323, State: Initialized, Role: FOLLOWER
I20260925 13:12:01.413643  5102 consensus_queue.cc:260] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER }
I20260925 13:12:01.413699  5102 raft_consensus.cc:388] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:12:01.413728  5102 raft_consensus.cc:482] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:12:01.413760  5102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:12:01.414299  5102 raft_consensus.cc:504] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER }
I20260925 13:12:01.414376  5102 leader_election.cc:302] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c232b126b78f4f9b8169b106478c3323; no voters: 
I20260925 13:12:01.414494  5102 leader_election.cc:288] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260925 13:12:01.414527  5105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:12:01.414641  5102 sys_catalog.cc:564] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:12:01.414695  5105 raft_consensus.cc:686] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 1 LEADER]: Becoming Leader. State: Replica: c232b126b78f4f9b8169b106478c3323, State: Running, Role: LEADER
I20260925 13:12:01.414795  5105 consensus_queue.cc:237] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER }
I20260925 13:12:01.414994  5105 sys_catalog.cc:455] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c232b126b78f4f9b8169b106478c3323. Latest consensus state: current_term: 1 leader_uuid: "c232b126b78f4f9b8169b106478c3323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER } }
I20260925 13:12:01.415020  5107 sys_catalog.cc:455] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c232b126b78f4f9b8169b106478c3323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c232b126b78f4f9b8169b106478c3323" member_type: VOTER } }
I20260925 13:12:01.415139  5105 sys_catalog.cc:458] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [sys.catalog]: This master's current role is: LEADER
I20260925 13:12:01.415159  5107 sys_catalog.cc:458] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [sys.catalog]: This master's current role is: LEADER
I20260925 13:12:01.415302  5114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:12:01.415470  5114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:12:01.415876   392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260925 13:12:01.416009  5114 catalog_manager.cc:1347] Generated new cluster ID: 998bf4151fbc43298182f10f2214be85
I20260925 13:12:01.416050  5114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:12:01.434433  5114 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:12:01.434953  5114 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:12:01.447150  5114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323: Generated new TSK 0
I20260925 13:12:01.447250  5114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:12:01.447803   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:12:01.449301  5123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.449301  5124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.449404  5126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.449537   392 server_base.cc:1034] running on GCE node
I20260925 13:12:01.449630   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:12:01.449653   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:12:01.449666   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341921449665 us; error 0 us; skew 500 ppm
I20260925 13:12:01.450460   392 webserver.cc:460] Webserver started at http://127.0.98.1:42403/ using document root <none> and password file <none>
I20260925 13:12:01.450572   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:12:01.450631   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:12:01.450680   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:12:01.451015   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root/instance:
uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.451999   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20260925 13:12:01.452517  5131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.452651   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:12:01.452716   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root
uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.452766   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:12:01.460461   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:12:01.460682   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:12:01.460882   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:12:01.461140   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:12:01.461175   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.461213   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:12:01.461242   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.464023   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:40643
I20260925 13:12:01.464059  5194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:40643 every 8 connection(s)
I20260925 13:12:01.464437   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:12:01.465929   392 server_base.cc:1034] running on GCE node
W20260925 13:12:01.465999  5202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.465909  5199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.466241  5200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.466444   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:12:01.466501   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:12:01.466519   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341921466519 us; error 0 us; skew 500 ppm
I20260925 13:12:01.467211   392 webserver.cc:460] Webserver started at http://127.0.98.2:42877/ using document root <none> and password file <none>
I20260925 13:12:01.467340   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:12:01.467393   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:12:01.467463   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:12:01.467707   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-1-root/instance:
uuid: "e296d16f88054aa98e7d701dc2c92ab4"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.468600  5195 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41763
I20260925 13:12:01.468621   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:12:01.468664  5195 heartbeater.cc:464] Registering TS with master...
I20260925 13:12:01.468760  5195 heartbeater.cc:511] Master 127.0.98.62:41763 requested a full tablet report, sending...
I20260925 13:12:01.469040  5066 ts_manager.cc:194] Registered new tserver with Master: 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:01.469244  5207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.469355   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20260925 13:12:01.469398   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-1-root
uuid: "e296d16f88054aa98e7d701dc2c92ab4"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.469434   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:12:01.469862  5066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60154
I20260925 13:12:01.483040   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:12:01.483258   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:12:01.483455   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:12:01.483669   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:12:01.483717   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.483757   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:12:01.483786   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.487360   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:32823
I20260925 13:12:01.487406  5270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:32823 every 8 connection(s)
I20260925 13:12:01.487798   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:12:01.489524  5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.489758  5274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.489768   392 server_base.cc:1034] running on GCE node
W20260925 13:12:01.490561  5277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.490746   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:12:01.490779   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:12:01.490792   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341921490792 us; error 0 us; skew 500 ppm
I20260925 13:12:01.491602   392 webserver.cc:460] Webserver started at http://127.0.98.3:36567/ using document root <none> and password file <none>
I20260925 13:12:01.491686   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:12:01.491755   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:12:01.491802   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:12:01.492168   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-2-root/instance:
uuid: "82d06f9d11244f5088d688a1c1e233df"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.492681  5271 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41763
I20260925 13:12:01.492744  5271 heartbeater.cc:464] Registering TS with master...
I20260925 13:12:01.492823  5271 heartbeater.cc:511] Master 127.0.98.62:41763 requested a full tablet report, sending...
I20260925 13:12:01.493067  5066 ts_manager.cc:194] Registered new tserver with Master: e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2:32823)
I20260925 13:12:01.493489  5066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60168
I20260925 13:12:01.493685   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:12:01.494252  5282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.494376   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:12:01.494428   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-2-root
uuid: "82d06f9d11244f5088d688a1c1e233df"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.494478   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:12:01.501181   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:12:01.501408   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:12:01.501595   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:12:01.501819   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:12:01.501870   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.501904   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:12:01.501931   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.505256   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:39425
I20260925 13:12:01.505324  5345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:39425 every 8 connection(s)
I20260925 13:12:01.505690   392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:12:01.507210  5349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.507269  5352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:12:01.507412  5350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:12:01.507980   392 server_base.cc:1034] running on GCE node
I20260925 13:12:01.508111   392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260925 13:12:01.508148   392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260925 13:12:01.508180   392 hybrid_clock.cc:648] HybridClock initialized: now 1790341921508180 us; error 0 us; skew 500 ppm
I20260925 13:12:01.509006   392 webserver.cc:460] Webserver started at http://127.0.98.4:40141/ using document root <none> and password file <none>
I20260925 13:12:01.509142   392 fs_manager.cc:359] Metadata directory not provided
I20260925 13:12:01.509194   392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:12:01.509246   392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:12:01.509688   392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-3-root/instance:
uuid: "68b4fe1ff96d4ebd95fe719e6602b480"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.509891  5346 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41763
I20260925 13:12:01.509949  5346 heartbeater.cc:464] Registering TS with master...
I20260925 13:12:01.510044  5346 heartbeater.cc:511] Master 127.0.98.62:41763 requested a full tablet report, sending...
I20260925 13:12:01.510308  5066 ts_manager.cc:194] Registered new tserver with Master: 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425)
I20260925 13:12:01.510670  5066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60174
I20260925 13:12:01.510823   392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260925 13:12:01.511353  5357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.511525   392 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:12:01.511577   392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-3-root
uuid: "68b4fe1ff96d4ebd95fe719e6602b480"
format_stamp: "Formatted at 2026-09-25 13:12:01 on dist-test-slave-2538"
I20260925 13:12:01.511627   392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:12:01.524451   392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260925 13:12:01.524751   392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:12:01.524952   392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:12:01.525228   392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:12:01.525277   392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.525319   392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:12:01.525345   392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.528520   392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.4:38791
I20260925 13:12:01.528605  5420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.4:38791 every 8 connection(s)
I20260925 13:12:01.531960  5421 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41763
I20260925 13:12:01.532032  5421 heartbeater.cc:464] Registering TS with master...
I20260925 13:12:01.532110  5421 heartbeater.cc:511] Master 127.0.98.62:41763 requested a full tablet report, sending...
I20260925 13:12:01.532364  5066 ts_manager.cc:194] Registered new tserver with Master: 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791)
I20260925 13:12:01.532747  5066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60188
I20260925 13:12:01.533155   392 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.004435783s
I20260925 13:12:01.538031   392 test_util.cc:268] Using random seed: 1984069613
I20260925 13:12:01.543627  5066 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60204:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260925 13:12:01.549207  5310 tablet_service.cc:1467] Processing CreateTablet for tablet 66d310a804d048af84e0c0730bd8f473 (DEFAULT_TABLE table=test-workload [id=f144bd52c54d4b29a972261ccc065ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:12:01.549368  5310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d310a804d048af84e0c0730bd8f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:12:01.549556  5385 tablet_service.cc:1467] Processing CreateTablet for tablet 66d310a804d048af84e0c0730bd8f473 (DEFAULT_TABLE table=test-workload [id=f144bd52c54d4b29a972261ccc065ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:12:01.549675  5385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d310a804d048af84e0c0730bd8f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:12:01.550019  5159 tablet_service.cc:1467] Processing CreateTablet for tablet 66d310a804d048af84e0c0730bd8f473 (DEFAULT_TABLE table=test-workload [id=f144bd52c54d4b29a972261ccc065ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:12:01.550120  5159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d310a804d048af84e0c0730bd8f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:12:01.550972  5442 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: Bootstrap starting.
I20260925 13:12:01.551502  5442 tablet_bootstrap.cc:654] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: Neither blocks nor log segments found. Creating new log.
I20260925 13:12:01.551786  5441 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: Bootstrap starting.
I20260925 13:12:01.551980  5444 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Bootstrap starting.
I20260925 13:12:01.552153  5442 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: No bootstrap required, opened a new log
I20260925 13:12:01.552248  5442 ts_tablet_manager.cc:1397] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:12:01.552402  5444 tablet_bootstrap.cc:654] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Neither blocks nor log segments found. Creating new log.
I20260925 13:12:01.552412  5442 raft_consensus.cc:348] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.552565  5442 raft_consensus.cc:374] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:12:01.552595  5442 raft_consensus.cc:729] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d06f9d11244f5088d688a1c1e233df, State: Initialized, Role: FOLLOWER
I20260925 13:12:01.552642  5442 consensus_queue.cc:260] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.552801  5441 tablet_bootstrap.cc:654] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: Neither blocks nor log segments found. Creating new log.
I20260925 13:12:01.552829  5346 heartbeater.cc:503] Master 127.0.98.62:41763 was elected leader, sending a full tablet report...
I20260925 13:12:01.552991  5444 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: No bootstrap required, opened a new log
I20260925 13:12:01.553056  5444 ts_tablet_manager.cc:1397] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Time spent bootstrapping tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:12:01.553234  5444 raft_consensus.cc:348] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.553284  5444 raft_consensus.cc:374] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:12:01.553326  5444 raft_consensus.cc:729] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3799b5f3ee46f4ba45fb7ece9a794e, State: Initialized, Role: FOLLOWER
I20260925 13:12:01.553371  5444 consensus_queue.cc:260] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.553474  5441 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: No bootstrap required, opened a new log
I20260925 13:12:01.553522  5441 ts_tablet_manager.cc:1397] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20260925 13:12:01.553536  5444 ts_tablet_manager.cc:1428] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.000s
I20260925 13:12:01.553568  5195 heartbeater.cc:503] Master 127.0.98.62:41763 was elected leader, sending a full tablet report...
I20260925 13:12:01.552806  5442 ts_tablet_manager.cc:1428] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20260925 13:12:01.553682  5441 raft_consensus.cc:348] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.553745  5441 raft_consensus.cc:374] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:12:01.553773  5441 raft_consensus.cc:729] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b4fe1ff96d4ebd95fe719e6602b480, State: Initialized, Role: FOLLOWER
I20260925 13:12:01.553838  5441 consensus_queue.cc:260] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.553993  5441 ts_tablet_manager.cc:1428] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: Time spent starting tablet: real 0.000s	user 0.000s	sys 0.001s
I20260925 13:12:01.554066  5421 heartbeater.cc:503] Master 127.0.98.62:41763 was elected leader, sending a full tablet report...
I20260925 13:12:01.708030  5448 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:12:01.708215  5448 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.708576  5448 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791), 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425)
I20260925 13:12:01.711704  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df" is_pre_election: true
I20260925 13:12:01.711851  5320 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3799b5f3ee46f4ba45fb7ece9a794e in term 0.
I20260925 13:12:01.711853  5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" is_pre_election: true
I20260925 13:12:01.712015  5395 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3799b5f3ee46f4ba45fb7ece9a794e in term 0.
I20260925 13:12:01.712137  5134 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3799b5f3ee46f4ba45fb7ece9a794e, 82d06f9d11244f5088d688a1c1e233df; no voters: 
I20260925 13:12:01.712231  5448 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:12:01.712278  5448 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:12:01.712304  5448 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:12:01.713032  5448 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.713217  5448 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [CANDIDATE]: Term 1 election: Requested vote from peers 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791), 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425)
I20260925 13:12:01.713353  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df"
I20260925 13:12:01.713439  5320 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:12:01.713532  5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b4fe1ff96d4ebd95fe719e6602b480"
I20260925 13:12:01.713601  5395 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:12:01.714012  5320 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3799b5f3ee46f4ba45fb7ece9a794e in term 1.
I20260925 13:12:01.714179  5134 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3799b5f3ee46f4ba45fb7ece9a794e, 82d06f9d11244f5088d688a1c1e233df; no voters: 
I20260925 13:12:01.714306  5395 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3799b5f3ee46f4ba45fb7ece9a794e in term 1.
I20260925 13:12:01.714315  5448 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:12:01.714564  5448 raft_consensus.cc:686] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 1 LEADER]: Becoming Leader. State: Replica: 5c3799b5f3ee46f4ba45fb7ece9a794e, State: Running, Role: LEADER
I20260925 13:12:01.714651  5448 consensus_queue.cc:237] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:01.715313  5065 catalog_manager.cc:5495] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1). New cstate: current_term: 1 leader_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } health_report { overall_health: HEALTHY } } }
I20260925 13:12:01.741569  5320 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Refusing update from remote peer 5c3799b5f3ee46f4ba45fb7ece9a794e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:12:01.741684  5395 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Refusing update from remote peer 5c3799b5f3ee46f4ba45fb7ece9a794e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:12:01.741787  5448 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:12:01.741928  5451 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:12:01.773679  5159 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:01.773790  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.773684  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.774575  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       1        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260925 13:12:01.786685   392 log_verifier.cc:126] Checking tablet 66d310a804d048af84e0c0730bd8f473
I20260925 13:12:01.803615   392 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 66d310a804d048af84e0c0730bd8f473
I20260925 13:12:01.803730   392 tablet_replica.cc:330] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: stopping tablet replica
I20260925 13:12:01.803761   392 raft_consensus.cc:2229] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:12:01.803814   392 raft_consensus.cc:2258] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:01.835187  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.835198  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.835538  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:01.836177  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C*             | 1            | -1           | Yes
 B             | A   B   C*             | 1            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:01.852344   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:02.033010  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.033123  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.033195  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.033358  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C*             | 1            | -1           | Yes
 B             | A   B   C*             | 1            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:02.044318   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:02.297951  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.299580  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.301177  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.301276  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C*             | 1            | -1           | Yes
 B             | A   B   C*             | 1            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:02.309701   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:02.494050  5271 heartbeater.cc:503] Master 127.0.98.62:41763 was elected leader, sending a full tablet report...
I20260925 13:12:02.673722  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.674319  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.674407  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:02.678821  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C*             | 1            | -1           | Yes
 B             | A   B   C*             | 1            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:02.684197   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:03.220597  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:03.220700  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:03.220794  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:03.221112  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C*             | 1            | -1           | Yes
 B             | A   B   C*             | 1            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:03.232337   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:03.244053  5632 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3799b5f3ee46f4ba45fb7ece9a794e)
I20260925 13:12:03.244200  5632 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:03.244503  5632 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:03.244776  5633 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3799b5f3ee46f4ba45fb7ece9a794e)
I20260925 13:12:03.244926  5633 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:03.245173  5633 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:03.248600  5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "82d06f9d11244f5088d688a1c1e233df" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" is_pre_election: true
I20260925 13:12:03.248595  5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "82d06f9d11244f5088d688a1c1e233df" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" is_pre_election: true
I20260925 13:12:03.248629  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df" is_pre_election: true
I20260925 13:12:03.248731  5395 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82d06f9d11244f5088d688a1c1e233df in term 1.
I20260925 13:12:03.248807  5320 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b4fe1ff96d4ebd95fe719e6602b480 in term 1.
I20260925 13:12:03.248965  5286 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b4fe1ff96d4ebd95fe719e6602b480, 82d06f9d11244f5088d688a1c1e233df; no voters: 
I20260925 13:12:03.248965  5360 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b4fe1ff96d4ebd95fe719e6602b480, 82d06f9d11244f5088d688a1c1e233df; no voters: 
W20260925 13:12:03.249096  5285 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:03.249150  5633 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:12:03.249205  5633 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3799b5f3ee46f4ba45fb7ece9a794e)
I20260925 13:12:03.249220  5632 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:12:03.249233  5633 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:12:03.249280  5632 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3799b5f3ee46f4ba45fb7ece9a794e)
I20260925 13:12:03.249382  5632 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:12:03.249459  5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" is_pre_election: true
W20260925 13:12:03.249622  5360 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:03.250193  5633 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:03.250198  5632 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:03.250324  5633 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 election: Requested vote from peers 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:03.250372  5632 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 election: Requested vote from peers 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:03.250486  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df"
I20260925 13:12:03.250492  5168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "82d06f9d11244f5088d688a1c1e233df" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e"
I20260925 13:12:03.250586  5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e"
I20260925 13:12:03.250562  5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "82d06f9d11244f5088d688a1c1e233df" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "68b4fe1ff96d4ebd95fe719e6602b480"
I20260925 13:12:03.250662  5395 raft_consensus.cc:2379] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 82d06f9d11244f5088d688a1c1e233df in current term 2: Already voted for candidate 68b4fe1ff96d4ebd95fe719e6602b480 in this term.
I20260925 13:12:03.250554  5320 raft_consensus.cc:2379] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 68b4fe1ff96d4ebd95fe719e6602b480 in current term 2: Already voted for candidate 82d06f9d11244f5088d688a1c1e233df in this term.
W20260925 13:12:03.250697  5285 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:03.250782  5286 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82d06f9d11244f5088d688a1c1e233df; no voters: 5c3799b5f3ee46f4ba45fb7ece9a794e, 68b4fe1ff96d4ebd95fe719e6602b480
I20260925 13:12:03.250864  5632 raft_consensus.cc:2726] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260925 13:12:03.250880  5360 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:03.250981  5360 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68b4fe1ff96d4ebd95fe719e6602b480; no voters: 5c3799b5f3ee46f4ba45fb7ece9a794e, 82d06f9d11244f5088d688a1c1e233df
I20260925 13:12:03.251039  5633 raft_consensus.cc:2726] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 13:12:04.019995  5385 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:04.020315  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:04.020406  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:04.020875  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       0        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B   C*             |              |              | Yes
 A             | A   B   C              | 2            | -1           | Yes
 B             | A   B   C              | 2            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:04.030479   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:05.047115  5686 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:12:05.047188  5686 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:05.047333  5686 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:05.047492  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df" is_pre_election: true
I20260925 13:12:05.047497  5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" is_pre_election: true
I20260925 13:12:05.047593  5320 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b4fe1ff96d4ebd95fe719e6602b480 in term 2.
W20260925 13:12:05.047715  5360 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:05.047777  5360 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68b4fe1ff96d4ebd95fe719e6602b480, 82d06f9d11244f5088d688a1c1e233df; no voters: 5c3799b5f3ee46f4ba45fb7ece9a794e
I20260925 13:12:05.047880  5686 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:12:05.047948  5686 raft_consensus.cc:482] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:12:05.047981  5686 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:12:05.048851  5686 raft_consensus.cc:504] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:05.049144  5686 leader_election.cc:288] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 election: Requested vote from peers 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425), 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643)
I20260925 13:12:05.049176  5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "82d06f9d11244f5088d688a1c1e233df"
I20260925 13:12:05.049243  5320 raft_consensus.cc:3037] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:12:05.049255  5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d310a804d048af84e0c0730bd8f473" candidate_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e"
W20260925 13:12:05.049397  5360 leader_election.cc:341] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Illegal state: cannot vote while shut down
I20260925 13:12:05.049809  5320 raft_consensus.cc:2454] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68b4fe1ff96d4ebd95fe719e6602b480 in term 3.
I20260925 13:12:05.049983  5360 leader_election.cc:302] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68b4fe1ff96d4ebd95fe719e6602b480, 82d06f9d11244f5088d688a1c1e233df; no voters: 5c3799b5f3ee46f4ba45fb7ece9a794e
I20260925 13:12:05.050128  5686 raft_consensus.cc:2781] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:12:05.050241  5686 raft_consensus.cc:686] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 LEADER]: Becoming Leader. State: Replica: 68b4fe1ff96d4ebd95fe719e6602b480, State: Running, Role: LEADER
I20260925 13:12:05.050294  5686 consensus_queue.cc:237] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } }
I20260925 13:12:05.050899  5062 catalog_manager.cc:5495] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 reported cstate change: term changed from 1 to 3, leader changed from 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1) to 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4). New cstate: current_term: 3 leader_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3799b5f3ee46f4ba45fb7ece9a794e" member_type: VOTER last_known_addr { host: "127.0.98.1" port: 40643 } health_report { overall_health: UNKNOWN } } }
I20260925 13:12:05.058911  5385 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:05.058982  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:05.059198  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:05.059703  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       1        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING [LEADER]
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): not running
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = 5c3799b5f3ee46f4ba45fb7ece9a794e
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | A*  B   C              | 3            | -1           | Yes
 B             | A   B   C              | 3            | -1           | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:05.069147   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:05.427445  5360 consensus_queue.cc:579] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: The tablet replica hosted on peer 5c3799b5f3ee46f4ba45fb7ece9a794e has failed
W20260925 13:12:05.427673  5360 consensus_peers.cc:487] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 -> Peer 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): Couldn't send request to peer 5c3799b5f3ee46f4ba45fb7ece9a794e. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:12:05.460431  5320 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Refusing update from remote peer 68b4fe1ff96d4ebd95fe719e6602b480: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260925 13:12:05.460768  5687 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:12:05.462503  5395 consensus_queue.cc:237] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } }
I20260925 13:12:05.462914  5320 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Refusing update from remote peer 68b4fe1ff96d4ebd95fe719e6602b480: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260925 13:12:05.463150  5687 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:12:05.463729  5687 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 LEADER]: Committing config change with OpId 3.7: config changed from index -1 to 7, VOTER 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } }
I20260925 13:12:05.463848  5320 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Committing config change with OpId 3.7: config changed from index -1 to 7, VOTER 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } }
I20260925 13:12:05.464835  5053 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 66d310a804d048af84e0c0730bd8f473 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260925 13:12:05.464872  5062 catalog_manager.cc:5495] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 reported cstate change: config changed from index -1 to 7, VOTER 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1) evicted. New cstate: current_term: 3 leader_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } health_report { overall_health: HEALTHY } } }
I20260925 13:12:05.465713  5159 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d310a804d048af84e0c0730bd8f473 with delete_type TABLET_DATA_TOMBSTONED (TS 5c3799b5f3ee46f4ba45fb7ece9a794e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40034
I20260925 13:12:05.465754  5395 consensus_queue.cc:237] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } }
I20260925 13:12:05.465946  5710 ts_tablet_manager.cc:1905] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260925 13:12:05.466235  5320 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Refusing update from remote peer 68b4fe1ff96d4ebd95fe719e6602b480: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20260925 13:12:05.466437  5687 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 13:12:05.466922  5686 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 LEADER]: Committing config change with OpId 3.8: config changed from index 7 to 8, NON_VOTER e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } } }
I20260925 13:12:05.467799  5710 ts_tablet_manager.cc:1918] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20260925 13:12:05.467830  5053 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 66d310a804d048af84e0c0730bd8f473 with cas_config_opid_index 7: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260925 13:12:05.467909  5710 log.cc:1196] T 66d310a804d048af84e0c0730bd8f473 P 5c3799b5f3ee46f4ba45fb7ece9a794e: Deleting WAL directory at /tmp/dist-test-taskKbs5sO/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790341876762790-392-0/minicluster-data/ts-0-root/wals/66d310a804d048af84e0c0730bd8f473
I20260925 13:12:05.467859  5062 catalog_manager.cc:5495] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 reported cstate change: config changed from index 7 to 8, NON_VOTER e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) added. New cstate: current_term: 3 leader_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260925 13:12:05.468075  5320 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Committing config change with OpId 3.8: config changed from index 7 to 8, NON_VOTER e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } } }
I20260925 13:12:05.468349  5052 catalog_manager.cc:4841] TS 5c3799b5f3ee46f4ba45fb7ece9a794e (127.0.98.1:40643): tablet 66d310a804d048af84e0c0730bd8f473 (table test-workload [id=f144bd52c54d4b29a972261ccc065ddd]) successfully deleted
W20260925 13:12:05.470002  5358 consensus_peers.cc:487] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 -> Peer e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2:32823): Couldn't send request to peer e296d16f88054aa98e7d701dc2c92ab4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66d310a804d048af84e0c0730bd8f473. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:12:06.021970  5713 ts_tablet_manager.cc:927] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Initiating tablet copy from peer 68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791)
I20260925 13:12:06.022156  5713 tablet_copy_client.cc:316] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: tablet copy: Beginning tablet copy session from remote peer at address 127.0.98.4:38791
I20260925 13:12:06.025506  5405 tablet_copy_service.cc:140] P 68b4fe1ff96d4ebd95fe719e6602b480: Received BeginTabletCopySession request for tablet 66d310a804d048af84e0c0730bd8f473 from peer e296d16f88054aa98e7d701dc2c92ab4 ({username='slave'} at 127.0.0.1:56468)
I20260925 13:12:06.025599  5405 tablet_copy_service.cc:161] P 68b4fe1ff96d4ebd95fe719e6602b480: Beginning new tablet copy session on tablet 66d310a804d048af84e0c0730bd8f473 from peer e296d16f88054aa98e7d701dc2c92ab4 at {username='slave'} at 127.0.0.1:56468: session id = e296d16f88054aa98e7d701dc2c92ab4-66d310a804d048af84e0c0730bd8f473
I20260925 13:12:06.026162  5405 tablet_copy_source_session.cc:205] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:12:06.026592  5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d310a804d048af84e0c0730bd8f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:12:06.027679  5713 tablet_copy_client.cc:791] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: tablet copy: Starting download of 0 data blocks...
I20260925 13:12:06.027812  5713 tablet_copy_client.cc:655] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: tablet copy: Starting download of 1 WAL segments...
I20260925 13:12:06.028632  5713 tablet_copy_client.cc:531] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:12:06.029372  5713 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Bootstrap starting.
I20260925 13:12:06.031378  5713 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:12:06.031626  5713 tablet_bootstrap.cc:492] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Bootstrap complete.
I20260925 13:12:06.031754  5713 ts_tablet_manager.cc:1397] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20260925 13:12:06.031915  5713 raft_consensus.cc:348] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } }
I20260925 13:12:06.031996  5713 raft_consensus.cc:729] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: e296d16f88054aa98e7d701dc2c92ab4, State: Initialized, Role: LEARNER
I20260925 13:12:06.032034  5713 consensus_queue.cc:260] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: NON_VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: true } }
I20260925 13:12:06.032210  5713 ts_tablet_manager.cc:1428] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: Time spent starting tablet: real 0.000s	user 0.002s	sys 0.000s
I20260925 13:12:06.032464  5405 tablet_copy_service.cc:342] P 68b4fe1ff96d4ebd95fe719e6602b480: Request end of tablet copy session e296d16f88054aa98e7d701dc2c92ab4-66d310a804d048af84e0c0730bd8f473 received from {username='slave'} at 127.0.0.1:56468
I20260925 13:12:06.032567  5405 tablet_copy_service.cc:433] P 68b4fe1ff96d4ebd95fe719e6602b480: ending tablet copy session e296d16f88054aa98e7d701dc2c92ab4-66d310a804d048af84e0c0730bd8f473 on tablet 66d310a804d048af84e0c0730bd8f473 with peer e296d16f88054aa98e7d701dc2c92ab4
I20260925 13:12:06.103812  5385 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:06.105762  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:06.107573  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:06.109332  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       1        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING [LEADER]
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2:32823): RUNNING [NONVOTER]
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = e296d16f88054aa98e7d701dc2c92ab4
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C~   |              |              | Yes
 A             | A*  B   C~   | 3            | 8            | Yes
 B             | A*  B   C~   | 3            | 8            | Yes
 C             | A   B   C~   | 3            | 8            | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:06.120484   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:06.645045  5245 raft_consensus.cc:1203] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.7->[3.8-3.8]   Dedup: 3.8->[]
I20260925 13:12:07.148016  5748 raft_consensus.cc:1050] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: attempting to promote NON_VOTER e296d16f88054aa98e7d701dc2c92ab4 to VOTER
I20260925 13:12:07.148202  5748 consensus_queue.cc:237] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false } }
I20260925 13:12:07.148268  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:07.149047  5320 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Refusing update from remote peer 68b4fe1ff96d4ebd95fe719e6602b480: Log matching property violated. Preceding OpId in replica: term: 3 index: 8. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20260925 13:12:07.149228  5245 raft_consensus.cc:1261] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 LEARNER]: Refusing update from remote peer 68b4fe1ff96d4ebd95fe719e6602b480: Log matching property violated. Preceding OpId in replica: term: 3 index: 8. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20260925 13:12:07.149322  5385 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:07.149595  5748 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:12:07.149705  5748 consensus_queue.cc:1035] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 13:12:07.150931  5782 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 LEADER]: Committing config change with OpId 3.9: config changed from index 8 to 9, e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false } } }
I20260925 13:12:07.152293  5063 catalog_manager.cc:5495] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 reported cstate change: config changed from index 8 to 9, e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260925 13:12:07.153160  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:07.153352  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:07.153309  5320 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Committing config change with OpId 3.9: config changed from index 8 to 9, e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false } } }
I20260925 13:12:07.153723  5245 raft_consensus.cc:2932] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 FOLLOWER]: Committing config change with OpId 3.9: config changed from index 8 to 9, e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "68b4fe1ff96d4ebd95fe719e6602b480" member_type: VOTER last_known_addr { host: "127.0.98.4" port: 38791 } } peers { permanent_uuid: "82d06f9d11244f5088d688a1c1e233df" member_type: VOTER last_known_addr { host: "127.0.98.3" port: 39425 } } peers { permanent_uuid: "e296d16f88054aa98e7d701dc2c92ab4" member_type: VOTER last_known_addr { host: "127.0.98.2" port: 32823 } attrs { promote: false } } }
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       1        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 66d310a804d048af84e0c0730bd8f473 of table 'test-workload' is conflicted: 3 replicas' active configs disagree with the leader master's
  68b4fe1ff96d4ebd95fe719e6602b480 (127.0.98.4:38791): RUNNING [LEADER]
  82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): RUNNING
  e296d16f88054aa98e7d701dc2c92ab4 (127.0.98.2:32823): RUNNING [NONVOTER]
All reported replicas are:
  A = 68b4fe1ff96d4ebd95fe719e6602b480
  B = 82d06f9d11244f5088d688a1c1e233df
  C = e296d16f88054aa98e7d701dc2c92ab4
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C~   |              |              | Yes
 A             | A*  B   C    | 3            | 9            | No
 B             | A*  B   C    | 3            | 9            | Yes
 C             | A*  B   C    | 3            | 9            | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20260925 13:12:07.162582   392 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260925 13:12:08.189725  5235 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:08.189956  5310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260925 13:12:08.190093  5385 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260925 13:12:08.195147  5159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address      | Status
----------------------------------+-------------------+---------
 c232b126b78f4f9b8169b106478c3323 | 127.0.98.62:41763 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |     Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+------------------+---------+----------+----------------+-----------------
 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | HEALTHY | <none>   |       0        |       0
 68b4fe1ff96d4ebd95fe719e6602b480 | 127.0.98.4:38791 | HEALTHY | <none>   |       1        |       0
 82d06f9d11244f5088d688a1c1e233df | 127.0.98.3:39425 | HEALTHY | <none>   |       0        |       0
 e296d16f88054aa98e7d701dc2c92ab4 | 127.0.98.2:32823 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.18.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |       Host       | Replica Count
-------+----------------------------------+------------------+---------------
 Small | 5c3799b5f3ee46f4ba45fb7ece9a794e | 127.0.98.1:40643 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260925 13:12:08.201538   392 log_verifier.cc:126] Checking tablet 66d310a804d048af84e0c0730bd8f473
I20260925 13:12:08.213330   392 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 66d310a804d048af84e0c0730bd8f473
I20260925 13:12:08.213850   392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260925 13:12:08.216975   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:08.229394   392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260925 13:12:08.230502   392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260925 13:12:08.234450   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:08.234592   392 tablet_replica.cc:330] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4: stopping tablet replica
I20260925 13:12:08.234668   392 raft_consensus.cc:2229] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:12:08.234732   392 raft_consensus.cc:2258] T 66d310a804d048af84e0c0730bd8f473 P e296d16f88054aa98e7d701dc2c92ab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:08.247517   392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260925 13:12:08.248484   392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260925 13:12:08.251739   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:08.251852   392 tablet_replica.cc:330] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df: stopping tablet replica
I20260925 13:12:08.251886   392 raft_consensus.cc:2229] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260925 13:12:08.251987   392 raft_consensus.cc:2258] T 66d310a804d048af84e0c0730bd8f473 P 82d06f9d11244f5088d688a1c1e233df [term 3 FOLLOWER]: Raft consensus is shut down!
W20260925 13:12:08.261848  5360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.98.3:39425: connect: Connection refused (error 111) [suppressed 16 similar messages]
W20260925 13:12:08.262301  5360 consensus_peers.cc:487] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 -> Peer 82d06f9d11244f5088d688a1c1e233df (127.0.98.3:39425): Couldn't send request to peer 82d06f9d11244f5088d688a1c1e233df. Status: Network error: Client connection negotiation failed: client connection to 127.0.98.3:39425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:12:08.264537   392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260925 13:12:08.265612   392 tablet_server.cc:178] TabletServer@127.0.98.4:0 shutting down...
I20260925 13:12:08.268855   392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260925 13:12:08.268996   392 tablet_replica.cc:330] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480: stopping tablet replica
I20260925 13:12:08.269048   392 raft_consensus.cc:2229] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 LEADER]: Raft consensus shutting down.
I20260925 13:12:08.269131   392 raft_consensus.cc:2258] T 66d310a804d048af84e0c0730bd8f473 P 68b4fe1ff96d4ebd95fe719e6602b480 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:08.281934   392 tablet_server.cc:194] TabletServer@127.0.98.4:0 shutdown complete.
I20260925 13:12:08.283360   392 master.cc:537] Master@127.0.98.62:41763 shutting down...
I20260925 13:12:08.286705   392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:12:08.286788   392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:12:08.286808   392 tablet_replica.cc:330] T 00000000000000000000000000000000 P c232b126b78f4f9b8169b106478c3323: stopping tablet replica
I20260925 13:12:08.298632   392 master.cc:552] Master@127.0.98.62:41763 shutdown complete.
[       OK ] FailedTabletsAreReplacedITest.OneReplica/1 (6914 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (10251 ms total)

[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (51527 ms total)
[  PASSED  ] 12 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TsTabletManagerITest.TestTableStats

 1 FAILED TEST
I20260925 13:12:08.303985   392 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~10 seconds ago
I20260925 13:12:08.304056   392 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:594: suppressed but not reported on 1 messages since previous log ~11 seconds ago
I20260925 13:12:08.304073   392 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:536: suppressed but not reported on 1 messages since previous log ~11 seconds ago
I20260925 13:12:08.304100   392 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 470 messages since previous log ~36 seconds ago
I20260925 13:12:08.304123   392 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 6 messages since previous log ~18 seconds ago