[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 18:02:20.252100 6506 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.096s user 0.458s sys 0.634s
W20260926 18:02:20.252754 6506 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.097s user 0.458s sys 0.638s
I20260926 18:02:20.256104 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:45755
I20260926 18:02:20.258152 6506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 18:02:20.259775 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:20.277736 6518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.278808 6517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.386199 6520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:20.386439 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:20.387647 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:20.387859 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:20.388051 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445740388028 us; error 0 us; skew 500 ppm
I20260926 18:02:20.388823 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:20.393388 6506 webserver.cc:460] Webserver started at http://127.6.90.190:38969/ using document root <none> and password file <none>
I20260926 18:02:20.394160 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:20.394346 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:20.394726 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:20.400442 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "56b4b63905194b29a86ae1b85c78099e"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.407526 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 18:02:20.411937 6525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.412961 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:20.413242 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "56b4b63905194b29a86ae1b85c78099e"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.413532 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:20.438313 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:20.440032 6506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 18:02:20.440461 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:20.509933 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:45755
I20260926 18:02:20.510002 6576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:45755 every 8 connection(s)
I20260926 18:02:20.515594 6577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:20.538116 6577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: Bootstrap starting.
I20260926 18:02:20.543875 6577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:20.545929 6577 log.cc:824] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: Log is configured to *not* fsync() on all Append() calls
I20260926 18:02:20.550235 6577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: No bootstrap required, opened a new log
I20260926 18:02:20.571807 6577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER }
I20260926 18:02:20.572309 6577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:20.572571 6577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b4b63905194b29a86ae1b85c78099e, State: Initialized, Role: FOLLOWER
I20260926 18:02:20.573346 6577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER }
I20260926 18:02:20.573860 6577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:20.574113 6577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:20.574431 6577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:20.578603 6577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER }
I20260926 18:02:20.579288 6577 leader_election.cc:302] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56b4b63905194b29a86ae1b85c78099e; no voters:
I20260926 18:02:20.580775 6577 leader_election.cc:288] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:20.581122 6580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:20.583973 6580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 1 LEADER]: Becoming Leader. State: Replica: 56b4b63905194b29a86ae1b85c78099e, State: Running, Role: LEADER
I20260926 18:02:20.584703 6580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER }
I20260926 18:02:20.585625 6577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:20.592308 6581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56b4b63905194b29a86ae1b85c78099e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER } }
I20260926 18:02:20.593079 6581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:20.592547 6582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56b4b63905194b29a86ae1b85c78099e. Latest consensus state: current_term: 1 leader_uuid: "56b4b63905194b29a86ae1b85c78099e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b4b63905194b29a86ae1b85c78099e" member_type: VOTER } }
I20260926 18:02:20.593456 6582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:20.596779 6587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:20.608772 6587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:20.613941 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:20.625222 6587 catalog_manager.cc:1347] Generated new cluster ID: 850ed324116f4eafa90dd7c5d034c36d
I20260926 18:02:20.625433 6587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:20.648532 6587 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:20.650331 6587 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:20.661417 6587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: Generated new TSK 0
I20260926 18:02:20.662225 6587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:20.680758 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:20.689153 6599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.691776 6600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.694229 6602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:20.695210 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:20.696024 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:20.696270 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:20.696472 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445740696458 us; error 0 us; skew 500 ppm
I20260926 18:02:20.697026 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:20.700814 6506 webserver.cc:460] Webserver started at http://127.6.90.129:41773/ using document root <none> and password file <none>
I20260926 18:02:20.701292 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:20.701457 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:20.701710 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:20.702929 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "366e3653ba5a4440988ecd53cce5a1ef"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.708854 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260926 18:02:20.713024 6607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.713784 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 18:02:20.714100 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "366e3653ba5a4440988ecd53cce5a1ef"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.714388 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:20.740239 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:20.741537 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:20.744637 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:20.749442 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:20.749661 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.749938 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:20.750124 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.849426 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:46705
I20260926 18:02:20.849494 6669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:46705 every 8 connection(s)
I20260926 18:02:20.855963 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:20.870551 6674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.871743 6675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:20.876811 6678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:20.877591 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:20.878381 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:20.878563 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:20.878754 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445740878738 us; error 0 us; skew 500 ppm
I20260926 18:02:20.879408 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:20.884389 6670 heartbeater.cc:347] Connected to a master server at 127.6.90.190:45755
I20260926 18:02:20.884842 6670 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:20.885913 6670 heartbeater.cc:511] Master 127.6.90.190:45755 requested a full tablet report, sending...
I20260926 18:02:20.889310 6542 ts_manager.cc:194] Registered new tserver with Master: 366e3653ba5a4440988ecd53cce5a1ef (127.6.90.129:46705)
I20260926 18:02:20.893173 6542 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36030
I20260926 18:02:20.894505 6506 webserver.cc:460] Webserver started at http://127.6.90.130:34555/ using document root <none> and password file <none>
I20260926 18:02:20.895118 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:20.895344 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:20.895648 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:20.897138 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "ecf27eac4f8d4293a87ab3f08083e5ad"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.903786 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260926 18:02:20.908303 6682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.909091 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:20.909335 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "ecf27eac4f8d4293a87ab3f08083e5ad"
format_stamp: "Formatted at 2026-09-26 18:02:20 on dist-test-slave-5fqf"
I20260926 18:02:20.909601 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:20.928364 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:20.929554 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:20.932330 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:20.935134 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:20.935300 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.935526 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:20.935663 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:20.996827 6670 heartbeater.cc:503] Master 127.6.90.190:45755 was elected leader, sending a full tablet report...
I20260926 18:02:21.030622 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:37533
I20260926 18:02:21.031212 6744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:37533 every 8 connection(s)
I20260926 18:02:21.050433 6745 heartbeater.cc:347] Connected to a master server at 127.6.90.190:45755
I20260926 18:02:21.050829 6745 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:21.051640 6745 heartbeater.cc:511] Master 127.6.90.190:45755 requested a full tablet report, sending...
I20260926 18:02:21.053385 6542 ts_manager.cc:194] Registered new tserver with Master: ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130:37533)
I20260926 18:02:21.053520 6506 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.018303688s
I20260926 18:02:21.055284 6542 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36036
I20260926 18:02:21.083175 6542 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260926 18:02:21.105296 6542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260926 18:02:21.158572 6745 heartbeater.cc:503] Master 127.6.90.190:45755 was elected leader, sending a full tablet report...
I20260926 18:02:21.172129 6710 tablet_service.cc:1467] Processing CreateTablet for tablet 2dff9e8582ab4f109ec19f3f1d95cd0e (DEFAULT_TABLE table=test-table [id=3feb563d67bc46f3aba7ae534af3f521]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:21.172263 6709 tablet_service.cc:1467] Processing CreateTablet for tablet dc3f9a9cc02b4a0da20610f8f3c158dd (DEFAULT_TABLE table=test-table [id=3feb563d67bc46f3aba7ae534af3f521]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:21.173498 6710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dff9e8582ab4f109ec19f3f1d95cd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:21.174081 6709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc3f9a9cc02b4a0da20610f8f3c158dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:21.176784 6635 tablet_service.cc:1467] Processing CreateTablet for tablet 2dff9e8582ab4f109ec19f3f1d95cd0e (DEFAULT_TABLE table=test-table [id=3feb563d67bc46f3aba7ae534af3f521]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:21.178512 6635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dff9e8582ab4f109ec19f3f1d95cd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:21.178825 6634 tablet_service.cc:1467] Processing CreateTablet for tablet dc3f9a9cc02b4a0da20610f8f3c158dd (DEFAULT_TABLE table=test-table [id=3feb563d67bc46f3aba7ae534af3f521]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:21.180019 6634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc3f9a9cc02b4a0da20610f8f3c158dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:21.201730 6759 tablet_bootstrap.cc:492] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: Bootstrap starting.
I20260926 18:02:21.205410 6758 tablet_bootstrap.cc:492] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: Bootstrap starting.
I20260926 18:02:21.207854 6759 tablet_bootstrap.cc:654] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:21.213675 6759 tablet_bootstrap.cc:492] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: No bootstrap required, opened a new log
I20260926 18:02:21.213994 6759 ts_tablet_manager.cc:1397] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260926 18:02:21.215960 6758 tablet_bootstrap.cc:654] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:21.215945 6759 raft_consensus.cc:348] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:21.216611 6759 raft_consensus.cc:729] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366e3653ba5a4440988ecd53cce5a1ef, State: Initialized, Role: FOLLOWER
I20260926 18:02:21.217034 6759 consensus_queue.cc:260] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:21.219239 6759 ts_tablet_manager.cc:1428] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 18:02:21.220007 6759 tablet_bootstrap.cc:492] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: Bootstrap starting.
I20260926 18:02:21.224649 6759 tablet_bootstrap.cc:654] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:21.231593 6758 tablet_bootstrap.cc:492] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: No bootstrap required, opened a new log
I20260926 18:02:21.232064 6758 ts_tablet_manager.cc:1397] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.012s
I20260926 18:02:21.234503 6758 raft_consensus.cc:348] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:21.235313 6758 raft_consensus.cc:729] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Initialized, Role: FOLLOWER
I20260926 18:02:21.235891 6758 consensus_queue.cc:260] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:21.243556 6758 ts_tablet_manager.cc:1428] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260926 18:02:21.244660 6758 tablet_bootstrap.cc:492] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: Bootstrap starting.
I20260926 18:02:21.245808 6759 tablet_bootstrap.cc:492] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: No bootstrap required, opened a new log
I20260926 18:02:21.246194 6759 ts_tablet_manager.cc:1397] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.017s
I20260926 18:02:21.248872 6759 raft_consensus.cc:348] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:21.249536 6759 raft_consensus.cc:729] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366e3653ba5a4440988ecd53cce5a1ef, State: Initialized, Role: FOLLOWER
I20260926 18:02:21.250025 6759 consensus_queue.cc:260] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:21.251101 6758 tablet_bootstrap.cc:654] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:21.252192 6759 ts_tablet_manager.cc:1428] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 18:02:21.260062 6758 tablet_bootstrap.cc:492] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: No bootstrap required, opened a new log
I20260926 18:02:21.260426 6758 ts_tablet_manager.cc:1397] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260926 18:02:21.262377 6758 raft_consensus.cc:348] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:21.262799 6758 raft_consensus.cc:729] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Initialized, Role: FOLLOWER
I20260926 18:02:21.263230 6758 consensus_queue.cc:260] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:21.265043 6758 ts_tablet_manager.cc:1428] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260926 18:02:21.286326 6746 tablet.cc:2369] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 18:02:21.305457 6506 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260926 18:02:21.305754 6506 raft_consensus.cc:412] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Emulating election...
I20260926 18:02:21.305970 6506 raft_consensus.cc:3037] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:21.309585 6506 raft_consensus.cc:686] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Becoming Leader. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:21.310187 6506 consensus_queue.cc:237] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:21.314579 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:21.319790 6542 catalog_manager.cc:5495] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad reported cstate change: term changed from 0 to 1, leader changed from <none> to ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130). New cstate: current_term: 1 leader_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: HEALTHY } } }
I20260926 18:02:21.340240 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:21.443547 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:21.646968 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:21.768640 6645 raft_consensus.cc:3037] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:21.773840 6764 consensus_queue.cc:1035] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:21.950405 6506 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260926 18:02:21.950619 6506 raft_consensus.cc:412] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Emulating election...
I20260926 18:02:21.950817 6506 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:21.955801 6506 raft_consensus.cc:686] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Becoming Leader. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:21.956487 6506 consensus_queue.cc:237] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:21.958456 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:21.962049 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:21.962121 6542 catalog_manager.cc:5495] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad reported cstate change: term changed from 0 to 1, leader changed from <none> to ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130). New cstate: current_term: 1 leader_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:22.065912 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:22.269418 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260926 18:02:22.506681 6645 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:22.511883 6764 consensus_queue.cc:1035] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:22.773168 6506 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260926 18:02:22.773391 6506 raft_consensus.cc:412] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Emulating election...
I20260926 18:02:22.773597 6506 raft_consensus.cc:3032] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:22.773774 6506 raft_consensus.cc:729] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:22.774227 6506 consensus_queue.cc:260] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:22.774943 6506 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:22.778533 6506 raft_consensus.cc:686] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 LEADER]: Becoming Leader. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:22.779098 6506 consensus_queue.cc:237] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:22.780288 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:22.783797 6540 catalog_manager.cc:5495] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:22.784420 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260926 18:02:22.887395 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260926 18:02:23.090495 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260926 18:02:23.393589 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260926 18:02:23.394378 6645 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:23.398550 6645 raft_consensus.cc:1261] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 2 FOLLOWER]: Refusing update from remote peer ecf27eac4f8d4293a87ab3f08083e5ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260926 18:02:23.399637 6764 consensus_queue.cc:1035] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:23.797215 6506 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260926 18:02:23.797439 6506 raft_consensus.cc:412] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 1 FOLLOWER]: Emulating election...
I20260926 18:02:23.797605 6506 raft_consensus.cc:3037] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:23.801095 6506 raft_consensus.cc:686] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 2 LEADER]: Becoming Leader. State: Replica: 366e3653ba5a4440988ecd53cce5a1ef, State: Running, Role: LEADER
I20260926 18:02:23.801565 6506 consensus_queue.cc:237] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:23.803386 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:23.806813 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260926 18:02:23.806730 6541 catalog_manager.cc:5495] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef reported cstate change: term changed from 1 to 2, leader changed from ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130) to 366e3653ba5a4440988ecd53cce5a1ef (127.6.90.129). New cstate: current_term: 2 leader_uuid: "366e3653ba5a4440988ecd53cce5a1ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:23.910392 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260926 18:02:24.113332 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260926 18:02:24.182950 6645 raft_consensus.cc:1226] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 2 LEADER]: Rejecting Update request from peer ecf27eac4f8d4293a87ab3f08083e5ad for earlier term 1. Current term is 2. Ops: []
I20260926 18:02:24.184293 6764 consensus_queue.cc:1046] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:24.185273 6764 consensus_queue.cc:1225] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Peer 366e3653ba5a4440988ecd53cce5a1ef log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260926 18:02:24.185614 6796 raft_consensus.cc:3032] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:24.185838 6796 raft_consensus.cc:729] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:24.186364 6796 consensus_queue.cc:260] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } } peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } }
I20260926 18:02:24.187071 6796 raft_consensus.cc:3037] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:24.326575 6720 raft_consensus.cc:1261] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 FOLLOWER]: Refusing update from remote peer 366e3653ba5a4440988ecd53cce5a1ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260926 18:02:24.327797 6797 consensus_queue.cc:1035] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:24.616766 6506 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260926 18:02:24.616992 6506 raft_consensus.cc:412] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 2 FOLLOWER]: Emulating election...
I20260926 18:02:24.617208 6506 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:24.620869 6506 raft_consensus.cc:686] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 3 LEADER]: Becoming Leader. State: Replica: 366e3653ba5a4440988ecd53cce5a1ef, State: Running, Role: LEADER
I20260926 18:02:24.621430 6506 consensus_queue.cc:237] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:24.623919 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:24.626251 6540 catalog_manager.cc:5495] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef reported cstate change: term changed from 2 to 3, leader changed from ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130) to 366e3653ba5a4440988ecd53cce5a1ef (127.6.90.129). New cstate: current_term: 3 leader_uuid: "366e3653ba5a4440988ecd53cce5a1ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: HEALTHY } } }
I20260926 18:02:24.627101 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260926 18:02:24.730130 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260926 18:02:24.933400 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260926 18:02:24.956620 6645 raft_consensus.cc:1226] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 3 LEADER]: Rejecting Update request from peer ecf27eac4f8d4293a87ab3f08083e5ad for earlier term 2. Current term is 3. Ops: []
I20260926 18:02:24.957715 6808 consensus_queue.cc:1046] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:24.958935 6808 consensus_queue.cc:1225] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Peer 366e3653ba5a4440988ecd53cce5a1ef log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260926 18:02:24.959367 6809 raft_consensus.cc:3032] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 LEADER]: Stepping down as leader of term 2
I20260926 18:02:24.959681 6809 raft_consensus.cc:729] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:24.960218 6809 consensus_queue.cc:260] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:24.961117 6809 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:25.073161 6720 raft_consensus.cc:1261] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 3 FOLLOWER]: Refusing update from remote peer 366e3653ba5a4440988ecd53cce5a1ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260926 18:02:25.074107 6797 consensus_queue.cc:1035] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:25.236330 6506 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260926 18:02:25.236548 6506 raft_consensus.cc:412] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 3 FOLLOWER]: Emulating election...
I20260926 18:02:25.236783 6506 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:25.240626 6506 raft_consensus.cc:686] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 4 LEADER]: Becoming Leader. State: Replica: ecf27eac4f8d4293a87ab3f08083e5ad, State: Running, Role: LEADER
I20260926 18:02:25.241145 6506 consensus_queue.cc:237] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:25.242321 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:25.245296 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260926 18:02:25.245162 6541 catalog_manager.cc:5495] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad reported cstate change: term changed from 3 to 4, leader changed from 366e3653ba5a4440988ecd53cce5a1ef (127.6.90.129) to ecf27eac4f8d4293a87ab3f08083e5ad (127.6.90.130). New cstate: current_term: 4 leader_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:25.348502 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260926 18:02:25.464768 6720 raft_consensus.cc:1226] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 4 LEADER]: Rejecting Update request from peer 366e3653ba5a4440988ecd53cce5a1ef for earlier term 3. Current term is 4. Ops: []
I20260926 18:02:25.465863 6797 consensus_queue.cc:1046] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260926 18:02:25.466372 6797 consensus_queue.cc:1225] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [LEADER]: Peer ecf27eac4f8d4293a87ab3f08083e5ad log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260926 18:02:25.466702 6811 raft_consensus.cc:3032] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 3 LEADER]: Stepping down as leader of term 3
I20260926 18:02:25.466931 6811 raft_consensus.cc:729] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 366e3653ba5a4440988ecd53cce5a1ef, State: Running, Role: LEADER
I20260926 18:02:25.467469 6811 consensus_queue.cc:260] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf27eac4f8d4293a87ab3f08083e5ad" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 37533 } } peers { permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 } }
I20260926 18:02:25.468308 6811 raft_consensus.cc:3037] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:25.551667 6506 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260926 18:02:25.730965 6645 raft_consensus.cc:1261] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 4 FOLLOWER]: Refusing update from remote peer ecf27eac4f8d4293a87ab3f08083e5ad: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260926 18:02:25.732028 6809 consensus_queue.cc:1035] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "366e3653ba5a4440988ecd53cce5a1ef" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260926 18:02:26.056274 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:26.073479 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:26.074012 6506 tablet_replica.cc:330] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef: stopping tablet replica
I20260926 18:02:26.074575 6506 raft_consensus.cc:2229] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:26.075093 6506 raft_consensus.cc:2258] T dc3f9a9cc02b4a0da20610f8f3c158dd P 366e3653ba5a4440988ecd53cce5a1ef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.076794 6506 tablet_replica.cc:330] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef: stopping tablet replica
I20260926 18:02:26.077169 6506 raft_consensus.cc:2229] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:26.077476 6506 raft_consensus.cc:2258] T 2dff9e8582ab4f109ec19f3f1d95cd0e P 366e3653ba5a4440988ecd53cce5a1ef [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.156152 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:26.163897 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:26.176411 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:26.176905 6506 tablet_replica.cc:330] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad: stopping tablet replica
I20260926 18:02:26.177412 6506 raft_consensus.cc:2229] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:26.177762 6506 raft_consensus.cc:2258] T dc3f9a9cc02b4a0da20610f8f3c158dd P ecf27eac4f8d4293a87ab3f08083e5ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.179098 6506 tablet_replica.cc:330] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad: stopping tablet replica
I20260926 18:02:26.179508 6506 raft_consensus.cc:2229] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 4 LEADER]: Raft consensus shutting down.
I20260926 18:02:26.179977 6506 raft_consensus.cc:2258] T 2dff9e8582ab4f109ec19f3f1d95cd0e P ecf27eac4f8d4293a87ab3f08083e5ad [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.197520 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:26.204886 6506 master.cc:537] Master@127.6.90.190:45755 shutting down...
I20260926 18:02:26.354528 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:26.355002 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.355284 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56b4b63905194b29a86ae1b85c78099e: stopping tablet replica
I20260926 18:02:26.371549 6506 master.cc:552] Master@127.6.90.190:45755 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7264 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260926 18:02:26.394183 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:41429
I20260926 18:02:26.395052 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:26.399530 6824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.400156 6825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.401580 6827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:26.402392 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:26.402971 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:26.403128 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:26.403262 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445746403250 us; error 0 us; skew 500 ppm
I20260926 18:02:26.403695 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:26.409672 6506 webserver.cc:460] Webserver started at http://127.6.90.190:40749/ using document root <none> and password file <none>
I20260926 18:02:26.410105 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:26.410266 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:26.410506 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:26.411427 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "82121766744349beb9116c7fc41cd4c9"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.415247 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 18:02:26.418035 6832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.418677 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 18:02:26.418910 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "82121766744349beb9116c7fc41cd4c9"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.419153 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:26.448215 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:26.449105 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:26.478987 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:41429
I20260926 18:02:26.479058 6883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:41429 every 8 connection(s)
I20260926 18:02:26.482254 6884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:26.490746 6884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9: Bootstrap starting.
I20260926 18:02:26.494246 6884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:26.497468 6884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9: No bootstrap required, opened a new log
I20260926 18:02:26.499019 6884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER }
I20260926 18:02:26.499351 6884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:26.499553 6884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82121766744349beb9116c7fc41cd4c9, State: Initialized, Role: FOLLOWER
I20260926 18:02:26.500032 6884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER }
I20260926 18:02:26.500478 6884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:26.500677 6884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:26.500900 6884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:26.504294 6884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER }
I20260926 18:02:26.504787 6884 leader_election.cc:302] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82121766744349beb9116c7fc41cd4c9; no voters:
I20260926 18:02:26.505712 6884 leader_election.cc:288] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:26.505968 6887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:26.507085 6887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 1 LEADER]: Becoming Leader. State: Replica: 82121766744349beb9116c7fc41cd4c9, State: Running, Role: LEADER
I20260926 18:02:26.507652 6887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER }
I20260926 18:02:26.508150 6884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:26.510191 6889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82121766744349beb9116c7fc41cd4c9. Latest consensus state: current_term: 1 leader_uuid: "82121766744349beb9116c7fc41cd4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER } }
I20260926 18:02:26.510280 6888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82121766744349beb9116c7fc41cd4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82121766744349beb9116c7fc41cd4c9" member_type: VOTER } }
I20260926 18:02:26.510792 6889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:26.510958 6888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:26.513572 6893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:26.518170 6893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:26.524569 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:26.526352 6893 catalog_manager.cc:1347] Generated new cluster ID: 5d84137cb5c5433cb578f04ddcd4fcf0
I20260926 18:02:26.526600 6893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:26.547871 6893 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:26.549417 6893 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:26.569429 6893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9: Generated new TSK 0
I20260926 18:02:26.570027 6893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:26.590533 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:26.596179 6905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.596267 6906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.597628 6908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:26.598557 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:26.599244 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:26.599406 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:26.599536 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445746599525 us; error 0 us; skew 500 ppm
I20260926 18:02:26.600015 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:26.602083 6506 webserver.cc:460] Webserver started at http://127.6.90.129:44271/ using document root <none> and password file <none>
I20260926 18:02:26.602521 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:26.602684 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:26.602900 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:26.603868 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "2b7798576e3246feb2b2f1d87226fa90"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.607676 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260926 18:02:26.610610 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.611234 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:26.611462 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "2b7798576e3246feb2b2f1d87226fa90"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.611706 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:26.629319 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:26.630206 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:26.631381 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:26.633276 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:26.633433 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.633651 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:26.633780 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.666354 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:36579
I20260926 18:02:26.666419 6975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:36579 every 8 connection(s)
I20260926 18:02:26.670136 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:26.676244 6980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.677850 6981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.678793 6983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:26.679193 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:26.679896 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:26.680081 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:26.680250 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445746680235 us; error 0 us; skew 500 ppm
I20260926 18:02:26.680766 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:26.682518 6976 heartbeater.cc:347] Connected to a master server at 127.6.90.190:41429
I20260926 18:02:26.682818 6976 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:26.683382 6976 heartbeater.cc:511] Master 127.6.90.190:41429 requested a full tablet report, sending...
I20260926 18:02:26.685112 6849 ts_manager.cc:194] Registered new tserver with Master: 2b7798576e3246feb2b2f1d87226fa90 (127.6.90.129:36579)
I20260926 18:02:26.686688 6849 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43522
I20260926 18:02:26.688994 6506 webserver.cc:460] Webserver started at http://127.6.90.130:46205/ using document root <none> and password file <none>
I20260926 18:02:26.689514 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:26.689672 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:26.689860 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:26.690837 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "933cec5fc397406bb43c16f033acd586"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.694712 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 18:02:26.697522 6988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.698158 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 18:02:26.698385 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "933cec5fc397406bb43c16f033acd586"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.698635 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:26.712543 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:26.713505 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:26.714690 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:26.716665 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:26.716833 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.717025 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:26.717160 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.749522 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:43907
I20260926 18:02:26.749605 7050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:43907 every 8 connection(s)
I20260926 18:02:26.761981 7051 heartbeater.cc:347] Connected to a master server at 127.6.90.190:41429
I20260926 18:02:26.762291 7051 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:26.762904 7051 heartbeater.cc:511] Master 127.6.90.190:41429 requested a full tablet report, sending...
I20260926 18:02:26.764544 6849 ts_manager.cc:194] Registered new tserver with Master: 933cec5fc397406bb43c16f033acd586 (127.6.90.130:43907)
I20260926 18:02:26.765126 6506 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.012828611s
I20260926 18:02:26.766340 6849 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43526
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260926 18:02:26.784674 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:26.789584 6976 heartbeater.cc:503] Master 127.6.90.190:41429 was elected leader, sending a full tablet report...
I20260926 18:02:26.799260 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:26.817227 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:26.824266 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:26.837692 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:26.852284 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:26.857764 6506 master.cc:537] Master@127.6.90.190:41429 shutting down...
I20260926 18:02:26.868606 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:26.869011 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:26.869349 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 82121766744349beb9116c7fc41cd4c9: stopping tablet replica
I20260926 18:02:26.885969 6506 master.cc:552] Master@127.6.90.190:41429 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (510 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260926 18:02:26.903952 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:42877
I20260926 18:02:26.904807 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:26.908752 7064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.909597 7065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:26.910897 7067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:26.910967 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:26.911697 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:26.911854 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:26.911957 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445746911948 us; error 0 us; skew 500 ppm
I20260926 18:02:26.912377 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:26.914283 6506 webserver.cc:460] Webserver started at http://127.6.90.190:38891/ using document root <none> and password file <none>
I20260926 18:02:26.914664 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:26.914795 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:26.914975 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:26.915844 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.919536 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 18:02:26.922312 7072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:26.922923 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:26.923146 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6"
format_stamp: "Formatted at 2026-09-26 18:02:26 on dist-test-slave-5fqf"
I20260926 18:02:26.923384 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:26.940321 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:26.941145 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:26.971625 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:42877
I20260926 18:02:26.971688 7123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:42877 every 8 connection(s)
I20260926 18:02:26.974884 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:26.983028 7124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6: Bootstrap starting.
I20260926 18:02:26.986455 7124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:26.989647 7124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6: No bootstrap required, opened a new log
I20260926 18:02:26.991185 7124 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER }
I20260926 18:02:26.991520 7124 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:26.991715 7124 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3fa8692d59f4a5fbb0b0ca72a110af6, State: Initialized, Role: FOLLOWER
I20260926 18:02:26.992182 7124 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER }
I20260926 18:02:26.992597 7124 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:26.992830 7124 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:26.993050 7124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:26.996316 7124 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER }
I20260926 18:02:26.996819 7124 leader_election.cc:302] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3fa8692d59f4a5fbb0b0ca72a110af6; no voters:
I20260926 18:02:26.997732 7124 leader_election.cc:288] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:26.997951 7127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:26.999069 7127 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 1 LEADER]: Becoming Leader. State: Replica: d3fa8692d59f4a5fbb0b0ca72a110af6, State: Running, Role: LEADER
I20260926 18:02:26.999630 7127 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER }
I20260926 18:02:27.000133 7124 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:27.002142 7129 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3fa8692d59f4a5fbb0b0ca72a110af6. Latest consensus state: current_term: 1 leader_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER } }
I20260926 18:02:27.002182 7128 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fa8692d59f4a5fbb0b0ca72a110af6" member_type: VOTER } }
I20260926 18:02:27.002698 7129 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:27.002861 7128 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:27.010434 7134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:27.014742 7134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:27.015630 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:27.022603 7134 catalog_manager.cc:1347] Generated new cluster ID: 9dd383c95ac84c6d87837843e4926c4e
I20260926 18:02:27.022827 7134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:27.040000 7134 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:27.041384 7134 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:27.055851 7134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6: Generated new TSK 0
I20260926 18:02:27.056401 7134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:27.081699 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:27.086603 7145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:27.087814 7146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:27.088464 7148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:27.088811 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:27.089496 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:27.089663 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:27.089811 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445747089798 us; error 0 us; skew 500 ppm
I20260926 18:02:27.090282 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:27.092207 6506 webserver.cc:460] Webserver started at http://127.6.90.129:40101/ using document root <none> and password file <none>
I20260926 18:02:27.092662 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:27.092816 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:27.093040 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:27.093953 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "d72be320fb8146b9aec2e02608631c6a"
format_stamp: "Formatted at 2026-09-26 18:02:27 on dist-test-slave-5fqf"
I20260926 18:02:27.097738 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 18:02:27.100477 7153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.101169 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 18:02:27.101409 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "d72be320fb8146b9aec2e02608631c6a"
format_stamp: "Formatted at 2026-09-26 18:02:27 on dist-test-slave-5fqf"
I20260926 18:02:27.101701 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:27.111032 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:27.112145 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:27.113349 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:27.115243 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:27.115402 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.115607 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:27.115731 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.149158 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:36867
I20260926 18:02:27.149246 7215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:36867 every 8 connection(s)
I20260926 18:02:27.152961 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:27.159099 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:27.160825 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:27.161898 7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:27.162714 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:27.163437 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:27.163674 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:27.163861 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445747163845 us; error 0 us; skew 500 ppm
I20260926 18:02:27.164520 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:27.165184 7216 heartbeater.cc:347] Connected to a master server at 127.6.90.190:42877
I20260926 18:02:27.165485 7216 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:27.166041 7216 heartbeater.cc:511] Master 127.6.90.190:42877 requested a full tablet report, sending...
I20260926 18:02:27.167006 6506 webserver.cc:460] Webserver started at http://127.6.90.130:38971/ using document root <none> and password file <none>
I20260926 18:02:27.167531 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:27.167750 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:27.167757 7089 ts_manager.cc:194] Registered new tserver with Master: d72be320fb8146b9aec2e02608631c6a (127.6.90.129:36867)
I20260926 18:02:27.168145 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:27.169445 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "e7a3d386e2144562b26dbf4c89a788d8"
format_stamp: "Formatted at 2026-09-26 18:02:27 on dist-test-slave-5fqf"
I20260926 18:02:27.170136 7089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60884
I20260926 18:02:27.173599 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 18:02:27.176304 7228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.176940 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:27.177162 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "e7a3d386e2144562b26dbf4c89a788d8"
format_stamp: "Formatted at 2026-09-26 18:02:27 on dist-test-slave-5fqf"
I20260926 18:02:27.177372 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:27.198379 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:27.199231 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:27.200511 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:27.202770 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:27.202934 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.203114 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:27.203238 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:27.235491 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:35201
I20260926 18:02:27.235555 7290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:35201 every 8 connection(s)
I20260926 18:02:27.248049 7291 heartbeater.cc:347] Connected to a master server at 127.6.90.190:42877
I20260926 18:02:27.248364 7291 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:27.248943 7291 heartbeater.cc:511] Master 127.6.90.190:42877 requested a full tablet report, sending...
I20260926 18:02:27.250465 7089 ts_manager.cc:194] Registered new tserver with Master: e7a3d386e2144562b26dbf4c89a788d8 (127.6.90.130:35201)
I20260926 18:02:27.250974 6506 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.012797625s
I20260926 18:02:27.252286 7089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60886
I20260926 18:02:27.270851 7089 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 18:02:27.272009 7089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260926 18:02:27.272553 7216 heartbeater.cc:503] Master 127.6.90.190:42877 was elected leader, sending a full tablet report...
I20260926 18:02:27.302331 7181 tablet_service.cc:1467] Processing CreateTablet for tablet 4d98004f9778430388c4e603f84148cf (DEFAULT_TABLE table=test-table [id=b1e2da631fc04f3591acb13d79a28801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:27.303277 7181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d98004f9778430388c4e603f84148cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:27.303205 7256 tablet_service.cc:1467] Processing CreateTablet for tablet 4d98004f9778430388c4e603f84148cf (DEFAULT_TABLE table=test-table [id=b1e2da631fc04f3591acb13d79a28801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:27.304154 7256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d98004f9778430388c4e603f84148cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:27.314217 7304 tablet_bootstrap.cc:492] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: Bootstrap starting.
I20260926 18:02:27.314982 7305 tablet_bootstrap.cc:492] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: Bootstrap starting.
I20260926 18:02:27.318224 7304 tablet_bootstrap.cc:654] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:27.318945 7305 tablet_bootstrap.cc:654] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:27.322388 7304 tablet_bootstrap.cc:492] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: No bootstrap required, opened a new log
I20260926 18:02:27.322754 7304 ts_tablet_manager.cc:1397] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260926 18:02:27.322788 7305 tablet_bootstrap.cc:492] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: No bootstrap required, opened a new log
I20260926 18:02:27.323194 7305 ts_tablet_manager.cc:1397] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260926 18:02:27.324836 7304 raft_consensus.cc:348] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:27.324987 7305 raft_consensus.cc:348] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:27.325371 7304 raft_consensus.cc:729] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Initialized, Role: FOLLOWER
I20260926 18:02:27.325671 7305 raft_consensus.cc:729] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a3d386e2144562b26dbf4c89a788d8, State: Initialized, Role: FOLLOWER
I20260926 18:02:27.325907 7304 consensus_queue.cc:260] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:27.326195 7305 consensus_queue.cc:260] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:27.329041 7291 heartbeater.cc:503] Master 127.6.90.190:42877 was elected leader, sending a full tablet report...
I20260926 18:02:27.329409 7304 ts_tablet_manager.cc:1428] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260926 18:02:27.329533 7305 ts_tablet_manager.cc:1428] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 18:02:27.361927 6506 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260926 18:02:27.362123 6506 raft_consensus.cc:412] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 0 FOLLOWER]: Emulating election...
I20260926 18:02:27.362314 6506 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:27.365876 6506 raft_consensus.cc:686] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 1 LEADER]: Becoming Leader. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Running, Role: LEADER
W20260926 18:02:27.366179 7216 heartbeater.cc:644] Failed to heartbeat to 127.6.90.190:42877 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260926 18:02:27.366492 6506 consensus_queue.cc:237] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:27.368923 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:27.387121 6506 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260926 18:02:27.490536 6506 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260926 18:02:27.693992 6506 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260926 18:02:27.793438 7266 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:27.798314 7308 consensus_queue.cc:1035] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:27.998526 6506 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260926 18:02:27.998742 6506 raft_consensus.cc:412] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 1 LEADER]: Emulating election...
I20260926 18:02:27.998939 6506 raft_consensus.cc:3032] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:27.999131 6506 raft_consensus.cc:729] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Running, Role: LEADER
I20260926 18:02:27.999533 6506 consensus_queue.cc:260] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:28.000160 6506 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:28.004197 6506 raft_consensus.cc:686] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 2 LEADER]: Becoming Leader. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Running, Role: LEADER
I20260926 18:02:28.004745 6506 consensus_queue.cc:237] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:28.005961 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:28.008913 6506 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260926 18:02:28.112413 6506 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260926 18:02:28.315976 6506 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260926 18:02:28.425050 7266 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:28.429189 7266 raft_consensus.cc:1261] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 2 FOLLOWER]: Refusing update from remote peer d72be320fb8146b9aec2e02608631c6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260926 18:02:28.430183 7308 consensus_queue.cc:1035] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:28.620427 6506 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260926 18:02:28.620638 6506 raft_consensus.cc:412] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 2 LEADER]: Emulating election...
I20260926 18:02:28.620834 6506 raft_consensus.cc:3032] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 2 LEADER]: Stepping down as leader of term 2
I20260926 18:02:28.621003 6506 raft_consensus.cc:729] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Running, Role: LEADER
I20260926 18:02:28.621397 6506 consensus_queue.cc:260] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:28.621979 6506 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:28.625792 6506 raft_consensus.cc:686] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 3 LEADER]: Becoming Leader. State: Replica: d72be320fb8146b9aec2e02608631c6a, State: Running, Role: LEADER
I20260926 18:02:28.626231 6506 consensus_queue.cc:237] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 } } peers { permanent_uuid: "d72be320fb8146b9aec2e02608631c6a" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36867 } }
I20260926 18:02:28.627406 6506 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260926 18:02:28.630236 6506 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260926 18:02:28.733492 6506 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260926 18:02:28.936908 6506 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260926 18:02:29.207598 7266 raft_consensus.cc:3037] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:29.212850 7266 raft_consensus.cc:1261] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 3 FOLLOWER]: Refusing update from remote peer d72be320fb8146b9aec2e02608631c6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260926 18:02:29.214108 7322 consensus_queue.cc:1035] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a3d386e2144562b26dbf4c89a788d8" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260926 18:02:29.244004 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:29.260145 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:29.260893 6506 tablet_replica.cc:330] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a: stopping tablet replica
I20260926 18:02:29.261394 6506 raft_consensus.cc:2229] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 3 LEADER]: Raft consensus shutting down.
I20260926 18:02:29.261884 6506 raft_consensus.cc:2258] T 4d98004f9778430388c4e603f84148cf P d72be320fb8146b9aec2e02608631c6a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:29.279359 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:29.287225 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:29.321509 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:29.322098 6506 tablet_replica.cc:330] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8: stopping tablet replica
I20260926 18:02:29.322559 6506 raft_consensus.cc:2229] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:29.322937 6506 raft_consensus.cc:2258] T 4d98004f9778430388c4e603f84148cf P e7a3d386e2144562b26dbf4c89a788d8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:29.331211 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:29.338099 6506 master.cc:537] Master@127.6.90.190:42877 shutting down...
I20260926 18:02:29.351270 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:29.351789 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:29.352079 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P d3fa8692d59f4a5fbb0b0ca72a110af6: stopping tablet replica
I20260926 18:02:29.368074 6506 master.cc:552] Master@127.6.90.190:42877 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2490 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10266 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260926 18:02:29.395938 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:38471
I20260926 18:02:29.396930 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:29.401875 7331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:29.402674 7332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:29.403991 7334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.404086 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:29.404808 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:29.404979 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:29.405120 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445749405106 us; error 0 us; skew 500 ppm
I20260926 18:02:29.405588 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:29.407719 6506 webserver.cc:460] Webserver started at http://127.6.90.190:33349/ using document root <none> and password file <none>
I20260926 18:02:29.408146 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:29.408294 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:29.408588 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:29.409618 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "805998e3af9c447c834493e33930df17"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.413560 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 18:02:29.416467 7339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.417085 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 18:02:29.417313 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "805998e3af9c447c834493e33930df17"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.417543 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:29.433739 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:29.434792 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:29.467945 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:38471
I20260926 18:02:29.468055 7390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:38471 every 8 connection(s)
I20260926 18:02:29.471562 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:29.480401 7391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17: Bootstrap starting.
I20260926 18:02:29.484148 7391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:29.487673 7391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17: No bootstrap required, opened a new log
I20260926 18:02:29.489591 7391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER }
I20260926 18:02:29.489989 7391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:29.490165 7391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805998e3af9c447c834493e33930df17, State: Initialized, Role: FOLLOWER
I20260926 18:02:29.490628 7391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER }
I20260926 18:02:29.491005 7391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:29.491178 7391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:29.491360 7391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:29.494932 7391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER }
I20260926 18:02:29.495362 7391 leader_election.cc:302] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805998e3af9c447c834493e33930df17; no voters:
I20260926 18:02:29.496263 7391 leader_election.cc:288] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:29.496604 7394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:29.497758 7394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 1 LEADER]: Becoming Leader. State: Replica: 805998e3af9c447c834493e33930df17, State: Running, Role: LEADER
I20260926 18:02:29.498273 7394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER }
I20260926 18:02:29.498845 7391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:29.500963 7396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805998e3af9c447c834493e33930df17. Latest consensus state: current_term: 1 leader_uuid: "805998e3af9c447c834493e33930df17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER } }
I20260926 18:02:29.502565 7396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:29.502084 7395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "805998e3af9c447c834493e33930df17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805998e3af9c447c834493e33930df17" member_type: VOTER } }
I20260926 18:02:29.503130 7395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:29.507808 7400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:29.513429 7400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:29.514288 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:29.521090 7400 catalog_manager.cc:1347] Generated new cluster ID: 5730b84e78624a78a2dace7bfd9347e1
I20260926 18:02:29.521343 7400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:29.540331 7400 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:29.541774 7400 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:29.556438 7400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17: Generated new TSK 0
I20260926 18:02:29.556982 7400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:29.580024 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:29.585062 7412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:29.586184 7413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.587908 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:29.588268 7415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.589108 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:29.589269 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:29.589377 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445749589368 us; error 0 us; skew 500 ppm
I20260926 18:02:29.589824 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:29.591796 6506 webserver.cc:460] Webserver started at http://127.6.90.129:36177/ using document root <none> and password file <none>
I20260926 18:02:29.592190 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:29.592330 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:29.592593 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:29.593667 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "753d9ead35c64c37beafbc7266bc4175"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.597607 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260926 18:02:29.600533 7420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.601151 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260926 18:02:29.601380 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "753d9ead35c64c37beafbc7266bc4175"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.601611 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:29.616904 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:29.617924 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:29.619268 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:29.621291 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:29.621464 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.621666 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:29.621814 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.656461 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:45045
I20260926 18:02:29.656517 7482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:45045 every 8 connection(s)
I20260926 18:02:29.660405 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:29.666965 7487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:29.669333 7488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.672667 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:29.673087 7490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.674031 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:29.674227 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:29.674386 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445749674372 us; error 0 us; skew 500 ppm
I20260926 18:02:29.674913 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:29.677259 6506 webserver.cc:460] Webserver started at http://127.6.90.130:39347/ using document root <none> and password file <none>
I20260926 18:02:29.677666 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:29.677829 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:29.678040 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:29.679065 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "2403edece9914b13bb332fc6b90bd51d"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.679315 7483 heartbeater.cc:347] Connected to a master server at 127.6.90.190:38471
I20260926 18:02:29.679637 7483 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:29.680243 7483 heartbeater.cc:511] Master 127.6.90.190:38471 requested a full tablet report, sending...
I20260926 18:02:29.682269 7356 ts_manager.cc:194] Registered new tserver with Master: 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045)
I20260926 18:02:29.684079 7356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40288
I20260926 18:02:29.685041 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 18:02:29.687981 7495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.688704 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 18:02:29.688939 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "2403edece9914b13bb332fc6b90bd51d"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.689239 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:29.707037 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:29.708068 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:29.709301 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:29.711303 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:29.711473 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.711680 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:29.711813 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.746238 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:44477
I20260926 18:02:29.746325 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:44477 every 8 connection(s)
I20260926 18:02:29.750300 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:29.756711 7561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:29.757834 7562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.760278 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:29.760592 7564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:29.761487 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:29.761721 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:29.761911 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445749761896 us; error 0 us; skew 500 ppm
I20260926 18:02:29.762550 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:29.762712 7558 heartbeater.cc:347] Connected to a master server at 127.6.90.190:38471
I20260926 18:02:29.763020 7558 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:29.763640 7558 heartbeater.cc:511] Master 127.6.90.190:38471 requested a full tablet report, sending...
I20260926 18:02:29.764971 6506 webserver.cc:460] Webserver started at http://127.6.90.131:42253/ using document root <none> and password file <none>
I20260926 18:02:29.765511 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:29.765563 7356 ts_manager.cc:194] Registered new tserver with Master: 2403edece9914b13bb332fc6b90bd51d (127.6.90.130:44477)
I20260926 18:02:29.765765 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:29.766072 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:29.767432 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-2-root/instance:
uuid: "808646dbfde848d084b6e36582d865bc"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.767493 7356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58800
I20260926 18:02:29.772042 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 18:02:29.774868 7569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.775480 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 18:02:29.775708 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-2-root
uuid: "808646dbfde848d084b6e36582d865bc"
format_stamp: "Formatted at 2026-09-26 18:02:29 on dist-test-slave-5fqf"
I20260926 18:02:29.775935 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1790445738975945-6506-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:29.793617 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:29.794591 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:29.795758 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:29.797672 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:29.797850 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.798048 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:29.798190 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:29.857218 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.131:35993
I20260926 18:02:29.857278 7631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.131:35993 every 8 connection(s)
I20260926 18:02:29.871799 7632 heartbeater.cc:347] Connected to a master server at 127.6.90.190:38471
I20260926 18:02:29.872080 7632 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:29.872658 7632 heartbeater.cc:511] Master 127.6.90.190:38471 requested a full tablet report, sending...
I20260926 18:02:29.874173 7356 ts_manager.cc:194] Registered new tserver with Master: 808646dbfde848d084b6e36582d865bc (127.6.90.131:35993)
I20260926 18:02:29.874775 6506 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013113404s
I20260926 18:02:29.875950 7356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58814
I20260926 18:02:29.898134 7356 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260926 18:02:29.899629 7356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 18:02:29.938448 7597 tablet_service.cc:1467] Processing CreateTablet for tablet 9818171c7dbf410eb2a4a02424b23605 (DEFAULT_TABLE table=test-table [id=164723246ed24dee98b6369afe4da1e2]), partition=
I20260926 18:02:29.939335 7597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9818171c7dbf410eb2a4a02424b23605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:29.943289 7448 tablet_service.cc:1467] Processing CreateTablet for tablet 9818171c7dbf410eb2a4a02424b23605 (DEFAULT_TABLE table=test-table [id=164723246ed24dee98b6369afe4da1e2]), partition=
I20260926 18:02:29.943748 7523 tablet_service.cc:1467] Processing CreateTablet for tablet 9818171c7dbf410eb2a4a02424b23605 (DEFAULT_TABLE table=test-table [id=164723246ed24dee98b6369afe4da1e2]), partition=
I20260926 18:02:29.944226 7448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9818171c7dbf410eb2a4a02424b23605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:29.944664 7523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9818171c7dbf410eb2a4a02424b23605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:29.958743 7647 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: Bootstrap starting.
I20260926 18:02:29.965328 7647 tablet_bootstrap.cc:654] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:29.965853 7649 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: Bootstrap starting.
I20260926 18:02:29.966713 7648 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: Bootstrap starting.
I20260926 18:02:29.970788 7649 tablet_bootstrap.cc:654] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:29.971683 7647 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: No bootstrap required, opened a new log
I20260926 18:02:29.972285 7648 tablet_bootstrap.cc:654] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:29.972582 7647 ts_tablet_manager.cc:1397] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260926 18:02:29.975164 7647 raft_consensus.cc:348] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.975906 7647 raft_consensus.cc:374] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:29.976200 7647 raft_consensus.cc:729] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808646dbfde848d084b6e36582d865bc, State: Initialized, Role: FOLLOWER
I20260926 18:02:29.976969 7647 consensus_queue.cc:260] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.979002 7649 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: No bootstrap required, opened a new log
I20260926 18:02:29.979389 7648 tablet_bootstrap.cc:492] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: No bootstrap required, opened a new log
I20260926 18:02:29.979499 7649 ts_tablet_manager.cc:1397] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260926 18:02:29.979884 7648 ts_tablet_manager.cc:1397] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: Time spent bootstrapping tablet: real 0.013s user 0.013s sys 0.000s
I20260926 18:02:29.979966 7632 heartbeater.cc:503] Master 127.6.90.190:38471 was elected leader, sending a full tablet report...
I20260926 18:02:29.980489 7647 ts_tablet_manager.cc:1428] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260926 18:02:29.981894 7649 raft_consensus.cc:348] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.982275 7648 raft_consensus.cc:348] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.982652 7649 raft_consensus.cc:374] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:29.982846 7648 raft_consensus.cc:374] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:29.982954 7649 raft_consensus.cc:729] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753d9ead35c64c37beafbc7266bc4175, State: Initialized, Role: FOLLOWER
I20260926 18:02:29.983132 7648 raft_consensus.cc:729] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403edece9914b13bb332fc6b90bd51d, State: Initialized, Role: FOLLOWER
I20260926 18:02:29.983716 7648 consensus_queue.cc:260] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.983753 7649 consensus_queue.cc:260] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:29.986747 7558 heartbeater.cc:503] Master 127.6.90.190:38471 was elected leader, sending a full tablet report...
I20260926 18:02:29.987605 7648 ts_tablet_manager.cc:1428] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260926 18:02:29.992493 7483 heartbeater.cc:503] Master 127.6.90.190:38471 was elected leader, sending a full tablet report...
I20260926 18:02:29.992962 7649 ts_tablet_manager.cc:1428] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260926 18:02:30.183456 7655 raft_consensus.cc:482] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:30.183861 7655 raft_consensus.cc:504] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:30.185778 7655 leader_election.cc:288] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 808646dbfde848d084b6e36582d865bc (127.6.90.131:35993), 2403edece9914b13bb332fc6b90bd51d (127.6.90.130:44477)
I20260926 18:02:30.198587 7607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "753d9ead35c64c37beafbc7266bc4175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808646dbfde848d084b6e36582d865bc" is_pre_election: true
I20260926 18:02:30.199508 7607 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753d9ead35c64c37beafbc7266bc4175 in term 0.
I20260926 18:02:30.199590 7533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "753d9ead35c64c37beafbc7266bc4175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403edece9914b13bb332fc6b90bd51d" is_pre_election: true
I20260926 18:02:30.200205 7533 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753d9ead35c64c37beafbc7266bc4175 in term 0.
I20260926 18:02:30.200388 7422 leader_election.cc:302] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753d9ead35c64c37beafbc7266bc4175, 808646dbfde848d084b6e36582d865bc; no voters:
I20260926 18:02:30.201062 7655 raft_consensus.cc:2781] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:30.201390 7655 raft_consensus.cc:482] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:30.201623 7655 raft_consensus.cc:3037] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:30.205924 7655 raft_consensus.cc:504] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:30.207216 7655 leader_election.cc:288] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [CANDIDATE]: Term 1 election: Requested vote from peers 808646dbfde848d084b6e36582d865bc (127.6.90.131:35993), 2403edece9914b13bb332fc6b90bd51d (127.6.90.130:44477)
I20260926 18:02:30.207836 7607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "753d9ead35c64c37beafbc7266bc4175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808646dbfde848d084b6e36582d865bc"
I20260926 18:02:30.208060 7533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "753d9ead35c64c37beafbc7266bc4175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403edece9914b13bb332fc6b90bd51d"
I20260926 18:02:30.208277 7607 raft_consensus.cc:3037] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:30.208571 7533 raft_consensus.cc:3037] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:30.212479 7607 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753d9ead35c64c37beafbc7266bc4175 in term 1.
I20260926 18:02:30.212800 7533 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753d9ead35c64c37beafbc7266bc4175 in term 1.
I20260926 18:02:30.213250 7422 leader_election.cc:302] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753d9ead35c64c37beafbc7266bc4175, 808646dbfde848d084b6e36582d865bc; no voters:
I20260926 18:02:30.213915 7655 raft_consensus.cc:2781] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:30.214636 7655 raft_consensus.cc:686] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 1 LEADER]: Becoming Leader. State: Replica: 753d9ead35c64c37beafbc7266bc4175, State: Running, Role: LEADER
I20260926 18:02:30.215292 7655 consensus_queue.cc:237] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:30.221216 7356 catalog_manager.cc:5495] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129). New cstate: current_term: 1 leader_uuid: "753d9ead35c64c37beafbc7266bc4175" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:30.710124 7655 consensus_queue.cc:1035] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:30.719981 7663 consensus_queue.cc:1035] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:31.389060 6506 tablet_replica.cc:330] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175: stopping tablet replica
I20260926 18:02:31.389544 6506 raft_consensus.cc:2229] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:31.390309 6506 raft_consensus.cc:2258] T 9818171c7dbf410eb2a4a02424b23605 P 753d9ead35c64c37beafbc7266bc4175 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 18:02:32.066239 7479 debug-util.cc:398] Leaking SignalData structure 0x7b0800118ba0 after lost signal to thread 6509
W20260926 18:02:32.066998 7479 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6120 after lost signal to thread 7390
W20260926 18:02:32.067719 7479 debug-util.cc:398] Leaking SignalData structure 0x7b080007b560 after lost signal to thread 7482
W20260926 18:02:32.068368 7479 debug-util.cc:398] Leaking SignalData structure 0x7b080002f880 after lost signal to thread 7557
W20260926 18:02:32.068969 7479 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6480 after lost signal to thread 7631
I20260926 18:02:32.640620 7670 raft_consensus.cc:482] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753d9ead35c64c37beafbc7266bc4175)
I20260926 18:02:32.641203 7670 raft_consensus.cc:504] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:32.641745 7671 raft_consensus.cc:482] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 753d9ead35c64c37beafbc7266bc4175)
I20260926 18:02:32.642220 7671 raft_consensus.cc:504] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:32.644421 7671 leader_election.cc:288] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045), 2403edece9914b13bb332fc6b90bd51d (127.6.90.130:44477)
I20260926 18:02:32.651752 7670 leader_election.cc:288] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 808646dbfde848d084b6e36582d865bc (127.6.90.131:35993), 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045)
I20260926 18:02:32.670430 7458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "808646dbfde848d084b6e36582d865bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753d9ead35c64c37beafbc7266bc4175" is_pre_election: true
W20260926 18:02:32.672099 7573 leader_election.cc:341] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045): Illegal state: cannot vote while shut down
I20260926 18:02:32.677577 7607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "2403edece9914b13bb332fc6b90bd51d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "808646dbfde848d084b6e36582d865bc" is_pre_election: true
I20260926 18:02:32.678200 7607 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2403edece9914b13bb332fc6b90bd51d in term 1.
I20260926 18:02:32.679692 7497 leader_election.cc:302] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403edece9914b13bb332fc6b90bd51d, 808646dbfde848d084b6e36582d865bc; no voters:
I20260926 18:02:32.680526 7670 raft_consensus.cc:2781] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 18:02:32.680830 7670 raft_consensus.cc:482] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 753d9ead35c64c37beafbc7266bc4175)
I20260926 18:02:32.681141 7670 raft_consensus.cc:3037] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:32.686646 7458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "2403edece9914b13bb332fc6b90bd51d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753d9ead35c64c37beafbc7266bc4175" is_pre_election: true
I20260926 18:02:32.687153 7670 raft_consensus.cc:504] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
W20260926 18:02:32.688033 7499 leader_election.cc:341] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045): Illegal state: cannot vote while shut down
I20260926 18:02:32.688849 7670 leader_election.cc:288] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 election: Requested vote from peers 808646dbfde848d084b6e36582d865bc (127.6.90.131:35993), 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045)
I20260926 18:02:32.689409 7607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "2403edece9914b13bb332fc6b90bd51d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "808646dbfde848d084b6e36582d865bc"
I20260926 18:02:32.689397 7533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "808646dbfde848d084b6e36582d865bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2403edece9914b13bb332fc6b90bd51d" is_pre_election: true
I20260926 18:02:32.689985 7607 raft_consensus.cc:3037] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:32.690279 7533 raft_consensus.cc:2379] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808646dbfde848d084b6e36582d865bc in current term 2: Already voted for candidate 2403edece9914b13bb332fc6b90bd51d in this term.
I20260926 18:02:32.692390 7458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9818171c7dbf410eb2a4a02424b23605" candidate_uuid: "2403edece9914b13bb332fc6b90bd51d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "753d9ead35c64c37beafbc7266bc4175"
I20260926 18:02:32.692801 7573 leader_election.cc:302] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808646dbfde848d084b6e36582d865bc; no voters: 2403edece9914b13bb332fc6b90bd51d, 753d9ead35c64c37beafbc7266bc4175
W20260926 18:02:32.694162 7499 leader_election.cc:341] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045): Illegal state: cannot vote while shut down
I20260926 18:02:32.694641 7607 raft_consensus.cc:2454] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403edece9914b13bb332fc6b90bd51d in term 2.
I20260926 18:02:32.694983 7671 raft_consensus.cc:2726] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 18:02:32.695585 7497 leader_election.cc:302] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2403edece9914b13bb332fc6b90bd51d, 808646dbfde848d084b6e36582d865bc; no voters: 753d9ead35c64c37beafbc7266bc4175
I20260926 18:02:32.696151 7670 raft_consensus.cc:2781] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:32.697374 7670 raft_consensus.cc:686] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 LEADER]: Becoming Leader. State: Replica: 2403edece9914b13bb332fc6b90bd51d, State: Running, Role: LEADER
I20260926 18:02:32.698477 7670 consensus_queue.cc:237] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 } } peers { permanent_uuid: "753d9ead35c64c37beafbc7266bc4175" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45045 } } peers { permanent_uuid: "2403edece9914b13bb332fc6b90bd51d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 44477 } }
I20260926 18:02:33.175863 7499 consensus_queue.cc:579] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [LEADER]: The tablet replica hosted on peer 753d9ead35c64c37beafbc7266bc4175 has failed
W20260926 18:02:33.176824 7499 consensus_peers.cc:487] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d -> Peer 753d9ead35c64c37beafbc7266bc4175 (127.6.90.129:45045): Couldn't send request to peer 753d9ead35c64c37beafbc7266bc4175. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:33.194100 7607 raft_consensus.cc:1261] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 2 FOLLOWER]: Refusing update from remote peer 2403edece9914b13bb332fc6b90bd51d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260926 18:02:33.195221 7670 consensus_queue.cc:1035] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "808646dbfde848d084b6e36582d865bc" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:33.425833 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:33.444886 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:33.788029 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:33.795171 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
W20260926 18:02:33.802693 7499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.90.129:45045: connect: Connection refused (error 111)
I20260926 18:02:33.812492 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:33.812943 6506 tablet_replica.cc:330] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d: stopping tablet replica
I20260926 18:02:33.813340 6506 raft_consensus.cc:2229] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:33.813938 6506 raft_consensus.cc:2258] T 9818171c7dbf410eb2a4a02424b23605 P 2403edece9914b13bb332fc6b90bd51d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:33.831228 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:33.837926 6506 tablet_server.cc:178] TabletServer@127.6.90.131:0 shutting down...
I20260926 18:02:33.852062 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:33.852556 6506 tablet_replica.cc:330] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc: stopping tablet replica
I20260926 18:02:33.852984 6506 raft_consensus.cc:2229] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:33.853330 6506 raft_consensus.cc:2258] T 9818171c7dbf410eb2a4a02424b23605 P 808646dbfde848d084b6e36582d865bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:33.870378 6506 tablet_server.cc:194] TabletServer@127.6.90.131:0 shutdown complete.
I20260926 18:02:33.877215 6506 master.cc:537] Master@127.6.90.190:38471 shutting down...
I20260926 18:02:33.889662 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:33.890084 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:33.890333 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 805998e3af9c447c834493e33930df17: stopping tablet replica
I20260926 18:02:33.906176 6506 master.cc:552] Master@127.6.90.190:38471 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (4531 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260926 18:02:33.926806 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:33885
I20260926 18:02:33.927695 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:33.931850 7691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:33.932924 7692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:33.933884 7694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:33.934580 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:33.935202 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:33.935395 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:33.935554 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445753935539 us; error 0 us; skew 500 ppm
I20260926 18:02:33.936039 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:33.938033 6506 webserver.cc:460] Webserver started at http://127.6.90.190:46777/ using document root <none> and password file <none>
I20260926 18:02:33.938453 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:33.938611 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:33.938862 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:33.939810 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "46ca853c3fda494a8031d57fa2cef74e"
format_stamp: "Formatted at 2026-09-26 18:02:33 on dist-test-slave-5fqf"
I20260926 18:02:33.943677 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260926 18:02:33.946462 7699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:33.947062 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:33.947294 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "46ca853c3fda494a8031d57fa2cef74e"
format_stamp: "Formatted at 2026-09-26 18:02:33 on dist-test-slave-5fqf"
I20260926 18:02:33.947538 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:33.984226 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:33.985164 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.016083 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:33885
I20260926 18:02:34.016182 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:33885 every 8 connection(s)
I20260926 18:02:34.020135 7751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:34.029021 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e: Bootstrap starting.
I20260926 18:02:34.032527 7751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:34.035772 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e: No bootstrap required, opened a new log
I20260926 18:02:34.037416 7751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER }
I20260926 18:02:34.037760 7751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:34.037955 7751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ca853c3fda494a8031d57fa2cef74e, State: Initialized, Role: FOLLOWER
I20260926 18:02:34.038440 7751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER }
I20260926 18:02:34.038854 7751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:34.039060 7751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:34.039284 7751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:34.042666 7751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER }
I20260926 18:02:34.043092 7751 leader_election.cc:302] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ca853c3fda494a8031d57fa2cef74e; no voters:
I20260926 18:02:34.044094 7751 leader_election.cc:288] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:34.044356 7754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:34.045531 7754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 1 LEADER]: Becoming Leader. State: Replica: 46ca853c3fda494a8031d57fa2cef74e, State: Running, Role: LEADER
I20260926 18:02:34.046113 7754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER }
I20260926 18:02:34.046589 7751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:34.050247 7756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ca853c3fda494a8031d57fa2cef74e. Latest consensus state: current_term: 1 leader_uuid: "46ca853c3fda494a8031d57fa2cef74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER } }
I20260926 18:02:34.050364 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ca853c3fda494a8031d57fa2cef74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ca853c3fda494a8031d57fa2cef74e" member_type: VOTER } }
I20260926 18:02:34.050772 7756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:34.050922 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:34.053108 7761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:34.059458 7761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:34.060596 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:34.066826 7761 catalog_manager.cc:1347] Generated new cluster ID: 57f6e3cf808f4976ab720f63f6009a61
I20260926 18:02:34.067070 7761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:34.087754 7761 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:34.089198 7761 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:34.098814 7761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e: Generated new TSK 0
I20260926 18:02:34.099392 7761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:34.126238 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:34.131235 7772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.132167 7773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.135569 7775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.136009 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.136757 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.136920 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.137084 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754137070 us; error 0 us; skew 500 ppm
I20260926 18:02:34.137573 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.139635 6506 webserver.cc:460] Webserver started at http://127.6.90.129:39113/ using document root <none> and password file <none>
I20260926 18:02:34.140064 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.140215 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.140466 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.141443 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "8438dd3b95fc4a2ca733539821854bbb"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.145397 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 18:02:34.148649 7780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.149366 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 18:02:34.149597 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "8438dd3b95fc4a2ca733539821854bbb"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.149823 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:34.158694 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.159583 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.160796 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:34.162745 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:34.162910 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.163112 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:34.163245 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.196610 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:37751
I20260926 18:02:34.196697 7842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:37751 every 8 connection(s)
I20260926 18:02:34.210592 7843 heartbeater.cc:347] Connected to a master server at 127.6.90.190:33885
I20260926 18:02:34.210875 7843 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:34.211431 7843 heartbeater.cc:511] Master 127.6.90.190:33885 requested a full tablet report, sending...
I20260926 18:02:34.212997 7716 ts_manager.cc:194] Registered new tserver with Master: 8438dd3b95fc4a2ca733539821854bbb (127.6.90.129:37751)
I20260926 18:02:34.213327 6506 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.013883761s
I20260926 18:02:34.214840 7716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54964
I20260926 18:02:34.232098 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:34.243934 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:34.258384 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:34.264708 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 18:02:34.267192 6506 tablet_server.cc:93] deduplicated master addresses: 127.6.90.190:33885
W20260926 18:02:34.270040 7853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.270918 7854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.271445 7856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.271831 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.272596 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.272758 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.272903 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754272890 us; error 0 us; skew 500 ppm
I20260926 18:02:34.273362 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.275435 6506 webserver.cc:460] Webserver started at http://127.6.90.129:39113/ using document root <none> and password file <none>
I20260926 18:02:34.275852 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.275990 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:34.279129 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:34.281303 7861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.281898 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 18:02:34.282156 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "8438dd3b95fc4a2ca733539821854bbb"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.282406 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:34.292996 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.293802 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.294986 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:34.296974 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:34.297175 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.297375 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:34.297505 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.331432 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:37751
I20260926 18:02:34.331524 7923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:37751 every 8 connection(s)
I20260926 18:02:34.337752 6506 tablet_server.cc:178] TabletServer@127.6.90.129:37751 shutting down...
I20260926 18:02:34.356387 7924 heartbeater.cc:347] Connected to a master server at 127.6.90.190:33885
I20260926 18:02:34.356818 7924 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:34.357576 7924 heartbeater.cc:511] Master 127.6.90.190:33885 requested a full tablet report, sending...
I20260926 18:02:34.359357 7716 ts_manager.cc:194] Re-registered known tserver with Master: 8438dd3b95fc4a2ca733539821854bbb (127.6.90.129:37751)
I20260926 18:02:34.360759 7716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54984
I20260926 18:02:34.363078 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:34.377562 6506 tablet_server.cc:194] TabletServer@127.6.90.129:37751 shutdown complete.
I20260926 18:02:34.383453 6506 master.cc:537] Master@127.6.90.190:33885 shutting down...
I20260926 18:02:34.393959 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:34.394371 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:34.394716 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 46ca853c3fda494a8031d57fa2cef74e: stopping tablet replica
I20260926 18:02:34.412096 6506 master.cc:552] Master@127.6.90.190:33885 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (503 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260926 18:02:34.430477 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:40281,127.6.90.189:35189,127.6.90.188:38505
I20260926 18:02:34.431523 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:34.435695 7931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.436690 7932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.438133 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:34.438524 7934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.439152 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.439312 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.439440 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754439430 us; error 0 us; skew 500 ppm
I20260926 18:02:34.439891 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.441893 6506 webserver.cc:460] Webserver started at http://127.6.90.190:34285/ using document root <none> and password file <none>
I20260926 18:02:34.442303 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.442451 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.442665 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.443579 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "e3ff4442fcb446c0a9eea4effe901f20"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.447456 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 18:02:34.450279 7939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.450912 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:34.451138 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "e3ff4442fcb446c0a9eea4effe901f20"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.451375 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:34.460923 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.461751 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.492221 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:40281
I20260926 18:02:34.492300 7990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:40281 every 8 connection(s)
I20260926 18:02:34.495450 7991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:34.495576 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:34.500663 7993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.500794 7994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.501662 7991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } has no permanent_uuid. Determining permanent_uuid...
W20260926 18:02:34.503551 7996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.503913 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.504588 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.504765 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.504923 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754504910 us; error 0 us; skew 500 ppm
I20260926 18:02:34.505388 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.507956 6506 webserver.cc:460] Webserver started at http://127.6.90.189:37511/ using document root <none> and password file <none>
I20260926 18:02:34.508528 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.508760 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.509037 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.510172 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root/instance:
uuid: "e8b9d58914a84061b815691412e4a3f2"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.514655 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 18:02:34.516371 7991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.518966 8003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.519821 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 18:02:34.520058 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
uuid: "e8b9d58914a84061b815691412e4a3f2"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.520291 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260926 18:02:34.520946 7991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.189:35189: Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111)
I20260926 18:02:34.556084 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.557029 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.559482 7991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } attempt: 1
W20260926 18:02:34.562897 7991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.189:35189: Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111)
I20260926 18:02:34.588716 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.189:35189
I20260926 18:02:34.588785 8055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.189:35189 every 8 connection(s)
I20260926 18:02:34.591949 8056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:34.592075 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:34.596799 8058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.597337 8059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.599103 8056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } has no permanent_uuid. Determining permanent_uuid...
W20260926 18:02:34.600050 8061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.600950 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.601872 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.602123 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.602308 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754602293 us; error 0 us; skew 500 ppm
I20260926 18:02:34.602882 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.603295 7991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } attempt: 2
I20260926 18:02:34.609942 6506 webserver.cc:460] Webserver started at http://127.6.90.188:41187/ using document root <none> and password file <none>
I20260926 18:02:34.610558 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.610805 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.611117 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.612432 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root/instance:
uuid: "f822563edfad492e8f52dc2d8f62d7da"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.613267 8056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.619367 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 18:02:34.621098 7991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.623382 8069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.624318 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 18:02:34.624684 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
uuid: "f822563edfad492e8f52dc2d8f62d7da"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.625015 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260926 18:02:34.625444 7991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.188:38505: Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:34.628088 8056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } has no permanent_uuid. Determining permanent_uuid...
W20260926 18:02:34.632711 8056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.188:38505: Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:34.641006 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.641825 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.660872 7991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } attempt: 1
W20260926 18:02:34.664736 7991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.188:38505: Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:34.666133 8056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } attempt: 1
W20260926 18:02:34.669751 8056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.90.188:38505: Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:34.675050 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.188:38505
I20260926 18:02:34.675134 8121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.188:38505 every 8 connection(s)
I20260926 18:02:34.677614 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:34.678414 8122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:34.683657 8122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.694532 8122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.703939 8122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } has no permanent_uuid. Determining permanent_uuid...
I20260926 18:02:34.708163 8056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } attempt: 2
I20260926 18:02:34.721128 8122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Bootstrap starting.
I20260926 18:02:34.724133 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Bootstrap starting.
I20260926 18:02:34.724896 8122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:34.728204 8056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:34.728982 8122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: No bootstrap required, opened a new log
I20260926 18:02:34.730721 8122 raft_consensus.cc:348] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.731200 8122 raft_consensus.cc:374] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:34.731419 8122 raft_consensus.cc:729] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f822563edfad492e8f52dc2d8f62d7da, State: Initialized, Role: FOLLOWER
I20260926 18:02:34.731794 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: No bootstrap required, opened a new log
I20260926 18:02:34.731961 8122 consensus_queue.cc:260] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.733494 8056 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.733994 8056 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:34.733732 8129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.734223 8056 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b9d58914a84061b815691412e4a3f2, State: Initialized, Role: FOLLOWER
I20260926 18:02:34.734495 8129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.735147 8122 sys_catalog.cc:564] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:34.734895 8056 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.736627 8130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.737386 8130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.738065 8056 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:34.740197 7991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } attempt: 2
W20260926 18:02:34.759485 8152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 18:02:34.759750 8152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 18:02:34.761354 7991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Bootstrap starting.
W20260926 18:02:34.762830 8149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 18:02:34.763065 8149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 18:02:34.765033 7991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:34.768287 7991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: No bootstrap required, opened a new log
I20260926 18:02:34.769855 7991 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.770323 7991 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:34.770531 7991 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ff4442fcb446c0a9eea4effe901f20, State: Initialized, Role: FOLLOWER
I20260926 18:02:34.770987 7991 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.772364 8154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.772876 8154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.773391 7991 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:34.781834 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260926 18:02:34.782076 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 18:02:34.783612 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:34.785530 8165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 18:02:34.785825 8165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 18:02:34.789608 8166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.790424 8167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.791723 8169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.792181 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.792855 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.793027 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.793181 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754793169 us; error 0 us; skew 500 ppm
I20260926 18:02:34.793663 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.795720 6506 webserver.cc:460] Webserver started at http://127.6.90.129:35183/ using document root <none> and password file <none>
I20260926 18:02:34.796139 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.796298 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.796589 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.797528 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.801362 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 18:02:34.804276 8174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.805008 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 18:02:34.805243 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.805467 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:34.809320 8129 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:34.809732 8129 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.811200 8129 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
I20260926 18:02:34.811875 7966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20" is_pre_election: true
I20260926 18:02:34.812074 8031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2" is_pre_election: true
I20260926 18:02:34.812486 7966 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 0.
I20260926 18:02:34.812680 8031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 0.
I20260926 18:02:34.812915 8154 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:34.813279 8154 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.813701 8073 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, f822563edfad492e8f52dc2d8f62d7da; no voters:
I20260926 18:02:34.814440 8129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:34.814723 8129 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:34.814982 8129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:34.814932 8154 leader_election.cc:288] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
I20260926 18:02:34.815552 8031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2" is_pre_election: true
I20260926 18:02:34.815927 8031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3ff4442fcb446c0a9eea4effe901f20 in term 0.
I20260926 18:02:34.816618 7940 leader_election.cc:302] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, e8b9d58914a84061b815691412e4a3f2; no voters:
I20260926 18:02:34.817128 8154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:34.817337 8154 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:34.817510 8154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:34.821486 8097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f822563edfad492e8f52dc2d8f62d7da" is_pre_election: true
I20260926 18:02:34.820742 8129 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.821659 8154 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.822674 8097 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e3ff4442fcb446c0a9eea4effe901f20 in current term 1: Already voted for candidate f822563edfad492e8f52dc2d8f62d7da in this term.
I20260926 18:02:34.823850 7966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20"
I20260926 18:02:34.823959 8154 leader_election.cc:288] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 1 election: Requested vote from peers e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
I20260926 18:02:34.824653 8031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2"
I20260926 18:02:34.824942 8097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f822563edfad492e8f52dc2d8f62d7da"
I20260926 18:02:34.825636 7966 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f822563edfad492e8f52dc2d8f62d7da in current term 1: Already voted for candidate e3ff4442fcb446c0a9eea4effe901f20 in this term.
I20260926 18:02:34.825784 8129 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 1 election: Requested vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
I20260926 18:02:34.825974 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2"
I20260926 18:02:34.825646 8097 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e3ff4442fcb446c0a9eea4effe901f20 in current term 1: Already voted for candidate f822563edfad492e8f52dc2d8f62d7da in this term.
I20260926 18:02:34.825124 8031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:34.827343 8070 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f822563edfad492e8f52dc2d8f62d7da; no voters: e3ff4442fcb446c0a9eea4effe901f20, e8b9d58914a84061b815691412e4a3f2
I20260926 18:02:34.827947 8129 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 18:02:34.829758 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:34.831019 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:34.832683 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:34.833046 8031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ff4442fcb446c0a9eea4effe901f20 in term 1.
I20260926 18:02:34.833935 7940 leader_election.cc:302] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, e8b9d58914a84061b815691412e4a3f2; no voters: f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:34.834640 8154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:34.835665 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:34.835915 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.835754 8154 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 LEADER]: Becoming Leader. State: Replica: e3ff4442fcb446c0a9eea4effe901f20, State: Running, Role: LEADER
I20260926 18:02:34.836174 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:34.836393 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:34.836573 8154 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:34.839946 8181 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff4442fcb446c0a9eea4effe901f20. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.840531 8181 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:34.842094 8184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:34.849192 8184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:34.858778 8031 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Refusing update from remote peer e3ff4442fcb446c0a9eea4effe901f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 18:02:34.858788 8097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Refusing update from remote peer e3ff4442fcb446c0a9eea4effe901f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 18:02:34.860373 8154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:34.860989 8181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:34.868438 8130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff4442fcb446c0a9eea4effe901f20. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.869166 8130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.874648 8215 mvcc.cc:205] Tried to move back new op lower bound from 7333665811888803840 to 7333665811816853504. Current Snapshot: MvccSnapshot[applied={T|T < 7333665811888803840}]
I20260926 18:02:34.881469 8130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.882164 8130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.881821 8154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.882483 8154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:34.883219 8184 catalog_manager.cc:1347] Generated new cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:34.883481 8184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:34.885695 8129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff4442fcb446c0a9eea4effe901f20. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.886364 8129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.888680 8154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.889285 8154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:34.899163 8129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ff4442fcb446c0a9eea4effe901f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:34.899837 8129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:34.921663 8184 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:34.923923 8184 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:34.930099 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:41673
I20260926 18:02:34.930377 8248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:41673 every 8 connection(s)
I20260926 18:02:34.955935 8184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Generated new TSK 0
I20260926 18:02:34.956719 8184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:34.963008 8249 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:34.963389 8249 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:34.964329 8249 heartbeater.cc:511] Master 127.6.90.188:38505 requested a full tablet report, sending...
I20260926 18:02:34.967725 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 18:02:34.970888 8250 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:34.971237 8250 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:34.971753 8087 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:34.972003 8250 heartbeater.cc:511] Master 127.6.90.189:35189 requested a full tablet report, sending...
I20260926 18:02:34.975396 8021 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
W20260926 18:02:34.981308 8256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:34.985059 8257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.985987 8251 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:34.986421 8251 heartbeater.cc:464] Registering TS with master...
W20260926 18:02:34.986913 8259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:34.987588 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:34.987561 8251 heartbeater.cc:511] Master 127.6.90.190:40281 requested a full tablet report, sending...
I20260926 18:02:34.988524 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:34.988783 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:34.988991 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445754988973 us; error 0 us; skew 500 ppm
I20260926 18:02:34.989503 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:34.989809 7956 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:34.991770 6506 webserver.cc:460] Webserver started at http://127.6.90.130:35399/ using document root <none> and password file <none>
I20260926 18:02:34.991889 7956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41460
I20260926 18:02:34.992205 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:34.992405 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:34.992619 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:34.993643 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:34.997512 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 18:02:35.000406 8264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.001035 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:35.001286 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:35.001530 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:35.040056 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:35.041219 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:35.042297 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:35.045189 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:35.045389 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.045608 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:35.045753 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.094569 8251 heartbeater.cc:503] Master 127.6.90.190:40281 was elected leader, sending a full tablet report...
I20260926 18:02:35.123872 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:38067
I20260926 18:02:35.123971 8326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:38067 every 8 connection(s)
I20260926 18:02:35.137949 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:35.154052 8334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:35.154701 8335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:35.162568 8327 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:35.162955 8327 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.164028 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:35.164024 8327 heartbeater.cc:511] Master 127.6.90.188:38505 requested a full tablet report, sending...
W20260926 18:02:35.165589 8338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:35.166354 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:35.166539 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:35.166527 8087 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:35.166824 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445755166813 us; error 0 us; skew 500 ppm
I20260926 18:02:35.167363 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:35.167510 8330 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:35.167835 8330 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.168629 8330 heartbeater.cc:511] Master 127.6.90.190:40281 requested a full tablet report, sending...
I20260926 18:02:35.169291 8328 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:35.169633 8328 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.170199 6506 webserver.cc:460] Webserver started at http://127.6.90.131:37935/ using document root <none> and password file <none>
I20260926 18:02:35.170347 8328 heartbeater.cc:511] Master 127.6.90.189:35189 requested a full tablet report, sending...
I20260926 18:02:35.170691 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:35.170696 7956 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:35.171041 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:35.171362 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:35.172526 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-2-root/instance:
uuid: "67a96471cc6b46e3ad4324475a2e472f"
format_stamp: "Formatted at 2026-09-26 18:02:35 on dist-test-slave-5fqf"
I20260926 18:02:35.172930 8021 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:35.173519 7956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41464
I20260926 18:02:35.177613 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 18:02:35.181733 8342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.182539 6506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 18:02:35.182765 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-2-root
uuid: "67a96471cc6b46e3ad4324475a2e472f"
format_stamp: "Formatted at 2026-09-26 18:02:35 on dist-test-slave-5fqf"
I20260926 18:02:35.182976 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:35.196551 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:35.197563 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:35.200105 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:35.202904 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:35.203173 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.203397 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:35.203519 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:35.275038 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.131:39391
I20260926 18:02:35.275103 8404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.131:39391 every 8 connection(s)
I20260926 18:02:35.276403 8330 heartbeater.cc:503] Master 127.6.90.190:40281 was elected leader, sending a full tablet report...
I20260926 18:02:35.295213 8405 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:35.295604 8405 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.296459 8405 heartbeater.cc:511] Master 127.6.90.188:38505 requested a full tablet report, sending...
I20260926 18:02:35.299161 8087 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.314466 8406 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:35.314774 8406 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.315462 8406 heartbeater.cc:511] Master 127.6.90.189:35189 requested a full tablet report, sending...
I20260926 18:02:35.317821 8021 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.319674 8408 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:35.319944 8408 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:35.320569 8408 heartbeater.cc:511] Master 127.6.90.190:40281 requested a full tablet report, sending...
I20260926 18:02:35.322093 7956 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.322594 6506 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014136318s
I20260926 18:02:35.323555 7956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41472
I20260926 18:02:35.372675 7956 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260926 18:02:35.374360 7956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 18:02:35.426291 8408 heartbeater.cc:503] Master 127.6.90.190:40281 was elected leader, sending a full tablet report...
I20260926 18:02:35.459632 8292 tablet_service.cc:1467] Processing CreateTablet for tablet 386c7447fb5543c9b6a5f27922017e76 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.461015 8292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c7447fb5543c9b6a5f27922017e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.461503 8369 tablet_service.cc:1467] Processing CreateTablet for tablet ea77d43447de400fa8864da7daab7bca (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.463290 8369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea77d43447de400fa8864da7daab7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.460701 8370 tablet_service.cc:1467] Processing CreateTablet for tablet 386c7447fb5543c9b6a5f27922017e76 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.466027 8290 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4824a63d564a36a20c87846dbd1c46 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.467412 8291 tablet_service.cc:1467] Processing CreateTablet for tablet ea77d43447de400fa8864da7daab7bca (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.468009 8204 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4824a63d564a36a20c87846dbd1c46 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.468811 8290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4824a63d564a36a20c87846dbd1c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.468539 8205 tablet_service.cc:1467] Processing CreateTablet for tablet ea77d43447de400fa8864da7daab7bca (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.469625 8206 tablet_service.cc:1467] Processing CreateTablet for tablet 386c7447fb5543c9b6a5f27922017e76 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.469849 8368 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4824a63d564a36a20c87846dbd1c46 (DEFAULT_TABLE table=test-table [id=399225e6572b4962b4ccc60d8e35e29e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:35.472115 8370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c7447fb5543c9b6a5f27922017e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.471369 8291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea77d43447de400fa8864da7daab7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.473217 8206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c7447fb5543c9b6a5f27922017e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.474442 8204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4824a63d564a36a20c87846dbd1c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.473495 8368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4824a63d564a36a20c87846dbd1c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.475196 8205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea77d43447de400fa8864da7daab7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:35.533252 8427 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: Bootstrap starting.
I20260926 18:02:35.534413 8428 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: Bootstrap starting.
I20260926 18:02:35.545368 8429 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:35.545729 8428 tablet_bootstrap.cc:654] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.547978 8427 tablet_bootstrap.cc:654] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.553076 8427 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: No bootstrap required, opened a new log
I20260926 18:02:35.553406 8427 ts_tablet_manager.cc:1397] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.004s
I20260926 18:02:35.554603 8429 tablet_bootstrap.cc:654] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.555431 8427 raft_consensus.cc:348] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.555929 8427 raft_consensus.cc:374] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.556111 8427 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.556742 8427 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.559659 8428 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: No bootstrap required, opened a new log
I20260926 18:02:35.560132 8428 ts_tablet_manager.cc:1397] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.000s
I20260926 18:02:35.561475 8429 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: No bootstrap required, opened a new log
I20260926 18:02:35.561853 8429 ts_tablet_manager.cc:1397] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260926 18:02:35.561381 8427 ts_tablet_manager.cc:1428] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260926 18:02:35.566057 8429 raft_consensus.cc:348] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.566768 8429 raft_consensus.cc:374] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.567029 8429 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.567641 8429 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.571379 8428 raft_consensus.cc:348] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.572144 8428 raft_consensus.cc:374] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.572504 8428 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.572912 8427 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Bootstrap starting.
I20260926 18:02:35.573303 8428 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.577540 8429 ts_tablet_manager.cc:1428] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260926 18:02:35.578464 8429 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:35.584060 8428 ts_tablet_manager.cc:1428] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: Time spent starting tablet: real 0.024s user 0.005s sys 0.003s
I20260926 18:02:35.590065 8428 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: Bootstrap starting.
I20260926 18:02:35.596527 8429 tablet_bootstrap.cc:654] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.600185 8428 tablet_bootstrap.cc:654] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.602309 8427 tablet_bootstrap.cc:654] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.607690 8433 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.608191 8433 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.609452 8429 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: No bootstrap required, opened a new log
I20260926 18:02:35.609942 8429 ts_tablet_manager.cc:1397] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.007s
I20260926 18:02:35.610570 8427 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: No bootstrap required, opened a new log
I20260926 18:02:35.610988 8427 ts_tablet_manager.cc:1397] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.000s
I20260926 18:02:35.612613 8429 raft_consensus.cc:348] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.613384 8428 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: No bootstrap required, opened a new log
I20260926 18:02:35.613545 8429 raft_consensus.cc:374] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.613920 8428 ts_tablet_manager.cc:1397] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.006s
I20260926 18:02:35.614037 8429 raft_consensus.cc:729] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.614529 8427 raft_consensus.cc:348] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.615159 8427 raft_consensus.cc:374] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.615479 8427 raft_consensus.cc:729] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.614969 8429 consensus_queue.cc:260] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.616072 8427 consensus_queue.cc:260] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.617296 8429 ts_tablet_manager.cc:1428] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260926 18:02:35.618345 8429 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:35.624313 8429 tablet_bootstrap.cc:654] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.629618 8434 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.630111 8434 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.632642 8434 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:35.635154 8433 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.645500 8433 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.645934 8433 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.648610 8433 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.653795 8435 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.654255 8435 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.657294 8428 raft_consensus.cc:348] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.658372 8428 raft_consensus.cc:374] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.658624 8428 raft_consensus.cc:729] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.659116 8435 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.663832 8428 consensus_queue.cc:260] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.665825 8428 ts_tablet_manager.cc:1428] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: Time spent starting tablet: real 0.052s user 0.004s sys 0.002s
I20260926 18:02:35.666798 8428 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: Bootstrap starting.
I20260926 18:02:35.678833 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.679538 8380 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 0.
I20260926 18:02:35.681829 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.682415 8380 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 0.
I20260926 18:02:35.683511 8267 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters:
I20260926 18:02:35.687320 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:35.688270 8219 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 0.
I20260926 18:02:35.689350 8267 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters:
I20260926 18:02:35.689875 8428 tablet_bootstrap.cc:654] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.687831 8433 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:35.690394 8433 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.690696 8433 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.692864 8344 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters:
I20260926 18:02:35.693128 8427 ts_tablet_manager.cc:1428] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent starting tablet: real 0.082s user 0.048s sys 0.032s
I20260926 18:02:35.694000 8435 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:35.695328 8435 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.695933 8435 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.702206 8429 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: No bootstrap required, opened a new log
I20260926 18:02:35.702642 8429 ts_tablet_manager.cc:1397] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.084s user 0.016s sys 0.021s
I20260926 18:02:35.703382 8439 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:35.703873 8439 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.704190 8439 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.705734 8427 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: Bootstrap starting.
I20260926 18:02:35.704702 8429 raft_consensus.cc:348] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.709679 8429 raft_consensus.cc:374] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.710740 8434 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.711609 8434 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.709936 8429 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.712966 8435 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.716149 8435 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 1 election: Requested vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.723106 8434 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.723546 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:35.724108 8219 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 0.
I20260926 18:02:35.710058 8433 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.726042 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:35.727080 8219 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.727392 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:35.728567 8433 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 election: Requested vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.731442 8380 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07f5d1825d624bd1ad7bc65fe13637b6 in current term 1: Already voted for candidate 67a96471cc6b46e3ad4324475a2e472f in this term.
I20260926 18:02:35.726011 8429 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.733990 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:35.734910 8216 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 0.
I20260926 18:02:35.732704 8434 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.737632 8434 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.740919 8427 tablet_bootstrap.cc:654] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:35.740743 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:35.745227 8379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.746316 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.747253 8379 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 0.
I20260926 18:02:35.747131 8266 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6; no voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:35.752003 8433 raft_consensus.cc:2726] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 18:02:35.751684 8380 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f in current term 1: Already voted for candidate 67a96471cc6b46e3ad4324475a2e472f in this term.
I20260926 18:02:35.754352 8177 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters:
I20260926 18:02:35.750872 8439 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.758033 8439 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 election: Requested vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.759835 8428 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: No bootstrap required, opened a new log
I20260926 18:02:35.765491 8428 ts_tablet_manager.cc:1397] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.010s
I20260926 18:02:35.768028 8428 raft_consensus.cc:348] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.768779 8428 raft_consensus.cc:374] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.769059 8149 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:35.769093 8428 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.765014 8152 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:35.769781 8152 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: loading cluster ID for follower catalog manager: success
I20260926 18:02:35.770388 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:35.771003 8380 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.752991 8434 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.769443 8149 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: loading cluster ID for follower catalog manager: success
I20260926 18:02:35.773329 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:35.760321 8219 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 1.
I20260926 18:02:35.773464 8434 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:35.777215 8216 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.773000 8428 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.784528 8300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:35.786113 8428 ts_tablet_manager.cc:1428] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260926 18:02:35.786859 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
W20260926 18:02:35.792815 8175 leader_election.cc:341] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067): Illegal state: must be running to vote when last-logged opid is not known
I20260926 18:02:35.793581 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:35.795552 8301 raft_consensus.cc:2379] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f in current term 1: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:35.796190 8302 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f in current term 1: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:35.805673 8429 ts_tablet_manager.cc:1428] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.103s user 0.022s sys 0.021s
W20260926 18:02:35.808904 8177 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.90.131:39391, user_credentials={real_user=slave}} blocked reactor thread for 59780.6us
I20260926 18:02:35.810196 8344 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters:
I20260926 18:02:35.811250 8177 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f
I20260926 18:02:35.811800 8435 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:35.817341 8379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.817925 8379 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 0.
I20260926 18:02:35.822944 8447 raft_consensus.cc:2726] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 18:02:35.820694 8380 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 1.
I20260926 18:02:35.824287 8177 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:35.825888 8267 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters:
I20260926 18:02:35.827793 8439 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:35.828234 8439 raft_consensus.cc:686] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 LEADER]: Becoming Leader. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Running, Role: LEADER
I20260926 18:02:35.829298 8439 consensus_queue.cc:237] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.830341 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:35.832163 8301 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67a96471cc6b46e3ad4324475a2e472f in current term 1: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:35.829820 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:35.833878 8446 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:35.834734 8446 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.835119 8302 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 67a96471cc6b46e3ad4324475a2e472f in current term 1: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:35.835103 8446 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.836575 8149 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: acquiring CA information for follower catalog manager: success
I20260926 18:02:35.837517 8152 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: acquiring CA information for follower catalog manager: success
I20260926 18:02:35.843698 8435 raft_consensus.cc:686] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 LEADER]: Becoming Leader. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:35.852535 8149 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 18:02:35.844913 8435 consensus_queue.cc:237] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.852805 8446 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.857964 8216 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 1.
I20260926 18:02:35.858417 8434 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.858258 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:35.859686 8446 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 election: Requested vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
W20260926 18:02:35.860291 8175 leader_election.cc:341] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067): Illegal state: must be running to vote when last-logged opid is not known
I20260926 18:02:35.858820 8434 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:35.860625 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:35.861477 8152 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 18:02:35.861222 8380 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.860975 7956 catalog_manager.cc:5495] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130). New cstate: current_term: 1 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:35.867843 8427 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: No bootstrap required, opened a new log
I20260926 18:02:35.868278 8427 ts_tablet_manager.cc:1397] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent bootstrapping tablet: real 0.163s user 0.015s sys 0.010s
I20260926 18:02:35.869609 8380 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 1.
I20260926 18:02:35.876556 8177 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:35.877277 8446 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:35.877737 8446 raft_consensus.cc:686] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 LEADER]: Becoming Leader. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Running, Role: LEADER
I20260926 18:02:35.877665 8427 raft_consensus.cc:348] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.878458 8427 raft_consensus.cc:374] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:35.871060 8434 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.878621 8446 consensus_queue.cc:237] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.880402 8427 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Initialized, Role: FOLLOWER
I20260926 18:02:35.881148 8427 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.883194 8427 ts_tablet_manager.cc:1428] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260926 18:02:35.886025 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:35.887929 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:35.887079 8434 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 2 election: Requested vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.889739 8380 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:35.898772 7955 catalog_manager.cc:5495] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f reported cstate change: term changed from 0 to 1, leader changed from <none> to 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131). New cstate: current_term: 1 leader_uuid: "67a96471cc6b46e3ad4324475a2e472f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:35.897135 8380 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 2.
I20260926 18:02:35.900633 8177 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:35.902120 8446 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:35.902843 8446 raft_consensus.cc:686] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 LEADER]: Becoming Leader. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Running, Role: LEADER
I20260926 18:02:35.903373 7954 catalog_manager.cc:5495] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129). New cstate: current_term: 1 leader_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:35.903997 8446 consensus_queue.cc:237] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.930222 7956 catalog_manager.cc:5495] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f reported cstate change: term changed from 1 to 2, leader changed from 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130) to 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129). New cstate: current_term: 2 leader_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:35.945034 8216 raft_consensus.cc:1226] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 LEADER]: Rejecting Update request from peer 07f5d1825d624bd1ad7bc65fe13637b6 for earlier term 1. Current term is 2. Ops: []
I20260926 18:02:35.946198 8439 consensus_queue.cc:1046] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:35.946733 8439 consensus_queue.cc:1225] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Peer 9a0e1006c4d94358929b16cbf0f4a32f log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260926 18:02:35.947113 8433 raft_consensus.cc:3032] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:35.947449 8433 raft_consensus.cc:729] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Running, Role: LEADER
I20260926 18:02:35.948454 8439 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.948071 8433 consensus_queue.cc:260] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.948892 8439 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:35.949254 8433 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:35.950680 8439 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:35.951772 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.951763 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:35.952595 8380 raft_consensus.cc:2379] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07f5d1825d624bd1ad7bc65fe13637b6 in current term 1: Already voted for candidate 9a0e1006c4d94358929b16cbf0f4a32f in this term.
I20260926 18:02:35.953476 8267 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6; no voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:35.954075 8439 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:35.957919 8439 raft_consensus.cc:2726] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 18:02:35.968546 8439 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:35.968902 8439 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.970317 8439 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.971112 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:35.971208 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:35.971776 8216 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 1.
I20260926 18:02:35.972572 8266 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 67a96471cc6b46e3ad4324475a2e472f
I20260926 18:02:35.973137 8439 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 18:02:35.973492 8439 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:35.973749 8439 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:35.977051 8435 consensus_queue.cc:1035] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:35.978159 8439 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.980110 8439 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 election: Requested vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:35.980660 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:35.981181 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:35.982561 8266 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6; no voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:35.983259 8439 raft_consensus.cc:2726] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 18:02:35.984154 8302 raft_consensus.cc:1226] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Rejecting Update request from peer 67a96471cc6b46e3ad4324475a2e472f for earlier term 1. Current term is 2. Ops: []
I20260926 18:02:35.985359 8435 consensus_queue.cc:1046] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:35.986856 8455 raft_consensus.cc:3032] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:35.987236 8455 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:35.987963 8455 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:35.988886 8455 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:35.999420 8446 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:36.004822 8446 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:36.022392 8301 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Refusing update from remote peer 9a0e1006c4d94358929b16cbf0f4a32f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260926 18:02:36.023272 8301 pending_rounds.cc:76] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Aborting all ops after (but not including) 0
I20260926 18:02:36.023495 8301 pending_rounds.cc:97] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260926 18:02:36.023790 8301 raft_consensus.cc:2866] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260926 18:02:36.030443 8434 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:36.047010 8380 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Refusing update from remote peer 9a0e1006c4d94358929b16cbf0f4a32f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 18:02:36.048391 8447 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:36.166908 8455 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:36.167433 8455 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:36.169247 8455 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:36.169906 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:36.170519 8302 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 2.
I20260926 18:02:36.170670 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:36.172425 8343 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters: 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:36.173226 8455 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 18:02:36.173626 8455 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:36.174180 8455 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:36.180018 8455 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:36.182518 8302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:36.183096 8302 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:36.189932 8447 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:36.190579 8455 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 election: Requested vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:36.191151 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:36.190451 8447 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:36.192116 8216 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Advancing to term 3
I20260926 18:02:36.192677 8302 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 3.
I20260926 18:02:36.193212 8447 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:36.193704 8343 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters:
I20260926 18:02:36.194638 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:36.195349 8301 raft_consensus.cc:2354] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f for earlier term 2. Current term is 3.
I20260926 18:02:36.195427 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:36.196710 8175 leader_election.cc:398] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067) with higher term. Message: Invalid argument: T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f for earlier term 2. Current term is 3.
I20260926 18:02:36.194618 8455 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader election won for term 3
I20260926 18:02:36.197367 8455 raft_consensus.cc:686] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 LEADER]: Becoming Leader. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:36.198159 8455 consensus_queue.cc:237] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:36.199463 8216 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 3.
I20260926 18:02:36.199862 8175 leader_election.cc:401] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260926 18:02:36.200685 8447 raft_consensus.cc:2726] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067) with higher term. Message: Invalid argument: T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f for earlier term 2. Current term is 3.
I20260926 18:02:36.206033 7953 catalog_manager.cc:5495] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "67a96471cc6b46e3ad4324475a2e472f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:36.293716 8216 raft_consensus.cc:1261] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Refusing update from remote peer 67a96471cc6b46e3ad4324475a2e472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260926 18:02:36.295102 8455 consensus_queue.cc:1035] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:36.304544 8301 raft_consensus.cc:1261] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Refusing update from remote peer 67a96471cc6b46e3ad4324475a2e472f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260926 18:02:36.306145 8435 consensus_queue.cc:1035] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:36.705881 8216 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46"
dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
from {username='slave'} at 127.0.0.1:41776
I20260926 18:02:36.706442 8216 raft_consensus.cc:580] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 LEADER]: Received request to step down
I20260926 18:02:36.706724 8216 raft_consensus.cc:3032] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 LEADER]: Stepping down as leader of term 1
I20260926 18:02:36.706920 8216 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Running, Role: LEADER
I20260926 18:02:36.707369 8216 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.708417 8216 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:36.860468 8455 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:36.860865 8455 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.862371 8455 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:36.863031 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:36.865325 8216 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 2.
I20260926 18:02:36.868260 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:36.870599 8344 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:36.872989 8455 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 18:02:36.873430 8455 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:36.873762 8455 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:36.879890 8455 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.881603 8455 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 2 election: Requested vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:36.882184 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:36.882371 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:36.888087 8216 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 2.
I20260926 18:02:36.889060 8344 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:36.889639 8455 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:36.890036 8455 raft_consensus.cc:686] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 LEADER]: Becoming Leader. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:36.890775 8455 consensus_queue.cc:237] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.892381 8485 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:36.892850 8485 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.894603 8485 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:36.895529 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:36.896231 8216 raft_consensus.cc:2379] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07f5d1825d624bd1ad7bc65fe13637b6 in current term 2: Already voted for candidate 67a96471cc6b46e3ad4324475a2e472f in this term.
I20260926 18:02:36.896960 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:36.898569 8266 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6; no voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:36.899896 8485 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:36.900290 7953 catalog_manager.cc:5495] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f reported cstate change: term changed from 1 to 2, leader changed from 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) to 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131). New cstate: current_term: 2 leader_uuid: "67a96471cc6b46e3ad4324475a2e472f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: HEALTHY } } }
I20260926 18:02:36.905153 8485 raft_consensus.cc:2726] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 18:02:36.935091 8219 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca"
dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
from {username='slave'} at 127.0.0.1:41776
I20260926 18:02:36.935554 8219 raft_consensus.cc:580] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 LEADER]: Received request to step down
I20260926 18:02:36.935902 8219 raft_consensus.cc:3032] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 LEADER]: Stepping down as leader of term 2
I20260926 18:02:36.936162 8219 raft_consensus.cc:729] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Running, Role: LEADER
I20260926 18:02:36.936823 8219 consensus_queue.cc:260] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:36.937708 8219 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:36.988448 8219 raft_consensus.cc:1261] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Refusing update from remote peer 67a96471cc6b46e3ad4324475a2e472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 18:02:36.989696 8455 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:36.997751 8301 raft_consensus.cc:1261] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Refusing update from remote peer 67a96471cc6b46e3ad4324475a2e472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 18:02:37.000187 8455 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:37.125070 8485 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:37.125520 8485 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.126957 8485 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:37.127712 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:37.127817 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:37.128235 8219 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 3.
I20260926 18:02:37.129110 8266 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 67a96471cc6b46e3ad4324475a2e472f
I20260926 18:02:37.129669 8485 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 18:02:37.129937 8485 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:37.130180 8485 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:37.131570 8492 raft_consensus.cc:482] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:37.132002 8492 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.133903 8492 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:37.135026 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:37.135665 8485 raft_consensus.cc:504] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.136973 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:37.137537 8485 leader_election.cc:288] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 election: Requested vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:37.137995 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:37.138206 8301 raft_consensus.cc:2379] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67a96471cc6b46e3ad4324475a2e472f in current term 3: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:37.138234 8219 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67a96471cc6b46e3ad4324475a2e472f in term 3.
I20260926 18:02:37.138434 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea77d43447de400fa8864da7daab7bca" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:37.139217 8380 raft_consensus.cc:3037] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:37.148495 8380 raft_consensus.cc:2454] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 3.
I20260926 18:02:37.150700 8267 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters: 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:37.152182 8344 leader_election.cc:302] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters:
I20260926 18:02:37.152814 8485 raft_consensus.cc:2781] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 18:02:37.152977 8492 raft_consensus.cc:2741] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260926 18:02:37.158154 8485 raft_consensus.cc:686] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 LEADER]: Becoming Leader. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Running, Role: LEADER
I20260926 18:02:37.159080 8485 consensus_queue.cc:237] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.166209 8380 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46"
dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
from {username='slave'} at 127.0.0.1:38410
I20260926 18:02:37.166762 8380 raft_consensus.cc:580] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 LEADER]: Received request to step down
I20260926 18:02:37.167069 8380 raft_consensus.cc:3032] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 LEADER]: Stepping down as leader of term 2
I20260926 18:02:37.167287 8380 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:37.167452 7956 catalog_manager.cc:5495] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 reported cstate change: term changed from 2 to 3, leader changed from 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) to 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130). New cstate: current_term: 3 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:37.167810 8380 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.169162 8380 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:37.245142 8380 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260926 18:02:37.246605 8494 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:37.247952 8219 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260926 18:02:37.249234 8494 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:37.279279 8446 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.280036 8446 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.281905 8446 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:37.282860 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:37.283164 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:37.283468 8380 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 3.
I20260926 18:02:37.284422 8177 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters:
I20260926 18:02:37.284988 8446 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 18:02:37.285254 8446 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.285467 8446 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:37.289106 8446 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.290410 8446 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 3 election: Requested vote from peers 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:37.290974 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6"
I20260926 18:02:37.291235 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:37.295559 8380 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0e1006c4d94358929b16cbf0f4a32f in term 3.
I20260926 18:02:37.296254 8177 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6
I20260926 18:02:37.296801 8446 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Leader election won for term 3
I20260926 18:02:37.297482 8446 raft_consensus.cc:686] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 LEADER]: Becoming Leader. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Running, Role: LEADER
I20260926 18:02:37.298139 8446 consensus_queue.cc:237] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.305919 7953 catalog_manager.cc:5495] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f reported cstate change: term changed from 2 to 3, leader changed from 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131) to 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129). New cstate: current_term: 3 leader_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:37.365207 8496 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.365610 8496 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:37.367244 8496 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:37.369241 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:37.369241 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:37.374688 8380 raft_consensus.cc:2379] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07f5d1825d624bd1ad7bc65fe13637b6 in current term 3: Already voted for candidate 9a0e1006c4d94358929b16cbf0f4a32f in this term.
I20260926 18:02:37.379076 8267 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6; no voters: 67a96471cc6b46e3ad4324475a2e472f, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:37.380088 8496 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:37.381194 8380 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76"
dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
from {username='slave'} at 127.0.0.1:38410
I20260926 18:02:37.381700 8380 raft_consensus.cc:580] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 LEADER]: Received request to step down
I20260926 18:02:37.382001 8380 raft_consensus.cc:3032] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 LEADER]: Stepping down as leader of term 3
I20260926 18:02:37.382287 8380 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 67a96471cc6b46e3ad4324475a2e472f, State: Running, Role: LEADER
I20260926 18:02:37.383145 8380 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:37.384326 8380 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:37.385967 8496 raft_consensus.cc:2726] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 18:02:37.386605 8301 raft_consensus.cc:1261] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Refusing update from remote peer 9a0e1006c4d94358929b16cbf0f4a32f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260926 18:02:37.388564 8446 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260926 18:02:37.397738 8380 raft_consensus.cc:1261] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Refusing update from remote peer 9a0e1006c4d94358929b16cbf0f4a32f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260926 18:02:37.399576 8446 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260926 18:02:37.524957 8485 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.525370 8485 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:37.526944 8485 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:37.527976 8216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" is_pre_election: true
I20260926 18:02:37.528156 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:37.528854 8380 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 4.
I20260926 18:02:37.529623 8267 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters: 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:37.530121 8485 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 18:02:37.530377 8485 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.530617 8485 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:37.534456 8485 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:37.535751 8485 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 election: Requested vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:37.536353 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:37.536646 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
I20260926 18:02:37.539120 8446 raft_consensus.cc:482] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 67a96471cc6b46e3ad4324475a2e472f)
I20260926 18:02:37.539533 8446 raft_consensus.cc:504] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:37.541862 8379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:37.542459 8446 leader_election.cc:288] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:37.541908 8380 raft_consensus.cc:2454] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 4.
I20260926 18:02:37.542016 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c7447fb5543c9b6a5f27922017e76" candidate_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
I20260926 18:02:37.543154 8301 raft_consensus.cc:2379] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0e1006c4d94358929b16cbf0f4a32f in current term 4: Already voted for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in this term.
I20260926 18:02:37.543572 8267 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters: 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:37.544445 8175 leader_election.cc:302] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a0e1006c4d94358929b16cbf0f4a32f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f
I20260926 18:02:37.544566 8485 raft_consensus.cc:2781] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 18:02:37.544996 8485 raft_consensus.cc:686] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 LEADER]: Becoming Leader. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Running, Role: LEADER
I20260926 18:02:37.545056 8446 raft_consensus.cc:3037] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:37.545936 8485 consensus_queue.cc:237] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:37.550100 8446 raft_consensus.cc:2726] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 18:02:37.553308 7953 catalog_manager.cc:5495] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 reported cstate change: term changed from 3 to 4, leader changed from 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131) to 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130). New cstate: current_term: 4 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:37.588145 7953 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41482:
table { table_id: "399225e6572b4962b4ccc60d8e35e29e" table_name: "test-table" } new_table_name: "NewTableName"
I20260926 18:02:37.610958 6506 master.cc:537] Master@127.6.90.190:40281 shutting down...
I20260926 18:02:37.625700 8219 raft_consensus.cc:1261] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260926 18:02:37.631736 8501 consensus_queue.cc:1035] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260926 18:02:37.634317 8380 raft_consensus.cc:1261] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260926 18:02:37.641265 8502 consensus_queue.cc:1035] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260926 18:02:37.651515 8504 tablet.cc:1722] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.676451 8505 tablet.cc:1722] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.683537 8508 tablet.cc:1722] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.697317 8510 tablet.cc:1722] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.731750 8511 tablet.cc:1722] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.756613 8510 tablet.cc:1722] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.757407 8512 tablet.cc:1722] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.766552 8512 tablet.cc:1722] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.778509 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:37.779454 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:37.779848 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: stopping tablet replica
I20260926 18:02:37.779980 8510 tablet.cc:1722] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:37.809263 6506 master.cc:552] Master@127.6.90.190:40281 shutdown complete.
I20260926 18:02:37.900892 8519 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e3ff4442fcb446c0a9eea4effe901f20)
I20260926 18:02:37.901417 8519 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:37.903247 8519 leader_election.cc:288] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
I20260926 18:02:37.904263 8097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8b9d58914a84061b815691412e4a3f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 16 } } ignore_live_leader: false dest_uuid: "f822563edfad492e8f52dc2d8f62d7da" is_pre_election: true
W20260926 18:02:37.908265 8008 leader_election.cc:334] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281): Network error: Client connection negotiation failed: client connection to 127.6.90.190:40281: connect: Connection refused (error 111)
I20260926 18:02:37.908682 8008 leader_election.cc:302] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8b9d58914a84061b815691412e4a3f2; no voters: e3ff4442fcb446c0a9eea4effe901f20, f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:37.909237 8519 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 18:02:37.970657 8522 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e3ff4442fcb446c0a9eea4effe901f20)
I20260926 18:02:37.971078 8522 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:37.972620 8522 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
I20260926 18:02:37.973526 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 2 candidate_status { last_received { term: 1 index: 16 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2" is_pre_election: true
I20260926 18:02:37.974022 8030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 1.
I20260926 18:02:37.974925 8070 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b9d58914a84061b815691412e4a3f2, f822563edfad492e8f52dc2d8f62d7da; no voters:
I20260926 18:02:37.975661 8522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 18:02:37.975986 8522 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Starting leader election (detected failure of leader e3ff4442fcb446c0a9eea4effe901f20)
I20260926 18:02:37.976233 8522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 1 FOLLOWER]: Advancing to term 2
W20260926 18:02:37.976867 8073 leader_election.cc:334] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281): Network error: Client connection negotiation failed: client connection to 127.6.90.190:40281: connect: Connection refused (error 111)
I20260926 18:02:37.980819 8522 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:37.982332 8522 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 election: Requested vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
I20260926 18:02:37.983198 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 2 candidate_status { last_received { term: 1 index: 16 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2"
I20260926 18:02:37.983742 8030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 1 FOLLOWER]: Advancing to term 2
W20260926 18:02:37.985147 8073 leader_election.cc:334] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281): Network error: Client connection negotiation failed: client connection to 127.6.90.190:40281: connect: Connection refused (error 111)
I20260926 18:02:37.988144 8030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 2.
I20260926 18:02:37.988916 8070 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e8b9d58914a84061b815691412e4a3f2, f822563edfad492e8f52dc2d8f62d7da; no voters: e3ff4442fcb446c0a9eea4effe901f20
I20260926 18:02:37.989480 8522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:37.990347 8522 raft_consensus.cc:686] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 LEADER]: Becoming Leader. State: Replica: f822563edfad492e8f52dc2d8f62d7da, State: Running, Role: LEADER
I20260926 18:02:37.990924 8522 consensus_queue.cc:237] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.16, Last appended by leader: 16, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:37.993978 8525 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:37.994498 8525 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:37.995654 8526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:38.001224 8526 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e]
I20260926 18:02:38.005913 8526 tablet_loader.cc:96] loaded metadata for tablet 386c7447fb5543c9b6a5f27922017e76 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.006790 8526 tablet_loader.cc:96] loaded metadata for tablet 3f4824a63d564a36a20c87846dbd1c46 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.007637 8526 tablet_loader.cc:96] loaded metadata for tablet ea77d43447de400fa8864da7daab7bca (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.008457 8526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:38.011242 8526 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:38.011425 8526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:38.015096 8526 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:38.018157 8526 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Loaded TSK: 0
I20260926 18:02:38.019369 8526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:38.047384 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:38.052307 8527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:38.053251 8528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:38.054595 8530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:38.055380 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:38.056169 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:38.056443 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:38.056627 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445758056610 us; error 0 us; skew 500 ppm
I20260926 18:02:38.057286 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:38.060218 6506 webserver.cc:460] Webserver started at http://127.6.90.190:34285/ using document root <none> and password file <none>
I20260926 18:02:38.060742 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:38.060928 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:38.064065 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:38.066463 8535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:38.067143 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:38.067408 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "e3ff4442fcb446c0a9eea4effe901f20"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:38.067665 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:38.087693 8030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Refusing update from remote peer f822563edfad492e8f52dc2d8f62d7da: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20260926 18:02:38.088953 8525 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260926 18:02:38.091670 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:38.091871 8519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.092631 8519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.092927 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:38.094184 8525 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
W20260926 18:02:38.094640 8073 consensus_peers.cc:487] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da -> Peer e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281): Couldn't send request to peer e3ff4442fcb446c0a9eea4effe901f20. Status: Network error: Client connection negotiation failed: client connection to 127.6.90.190:40281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:38.095629 8525 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:38.096505 8519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.097251 8519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.133042 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:40281
I20260926 18:02:38.133143 8587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:40281 every 8 connection(s)
I20260926 18:02:38.139183 8249 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:38.142674 8588 sys_catalog.cc:263] Verifying existing consensus state
I20260926 18:02:38.145823 8327 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:38.148825 8588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Bootstrap starting.
I20260926 18:02:38.177819 8408 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:38.181458 8251 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:38.192781 8330 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:38.191790 8086 catalog_manager.cc:6040] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: NewTableName [id=399225e6572b4962b4ccc60d8e35e29e] alter complete (version 1)
I20260926 18:02:38.223377 8405 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:38.238639 6506 master.cc:537] Master@127.6.90.188:38505 shutting down...
I20260926 18:02:38.239881 8588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:38.240939 8588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: Bootstrap complete.
I20260926 18:02:38.243650 8588 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.244396 8588 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ff4442fcb446c0a9eea4effe901f20, State: Initialized, Role: FOLLOWER
I20260926 18:02:38.245078 8588 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 1.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.261781 8588 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:38.262392 8563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:38.293776 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:38.294793 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:38.295378 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: stopping tablet replica
I20260926 18:02:38.299567 8596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.300278 8596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.302935 8610 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.302827 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 2 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.303606 8610 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.303821 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.327703 8612 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:38.327925 8612 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: loading cluster ID for follower catalog manager: success
I20260926 18:02:38.332659 8612 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: acquiring CA information for follower catalog manager: success
I20260926 18:02:38.337227 8612 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 18:02:38.339762 6506 master.cc:552] Master@127.6.90.188:38505 shutdown complete.
I20260926 18:02:38.354696 8552 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" instance_seqno: 1790445754907997) as {username='slave'} at 127.0.0.1:41704; Asking this server to re-register.
I20260926 18:02:38.355804 8251 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.358759 8552 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" instance_seqno: 1790445755257675) as {username='slave'} at 127.0.0.1:41700; Asking this server to re-register.
I20260926 18:02:38.362988 8408 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.363144 8553 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:38.364923 8552 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" instance_seqno: 1790445755106376) as {username='slave'} at 127.0.0.1:41706; Asking this server to re-register.
I20260926 18:02:38.366087 8330 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.366595 8552 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:38.368556 8552 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:38.474053 8519 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f822563edfad492e8f52dc2d8f62d7da)
I20260926 18:02:38.474475 8519 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.475824 8519 leader_election.cc:288] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
I20260926 18:02:38.476979 8600 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f822563edfad492e8f52dc2d8f62d7da)
I20260926 18:02:38.477372 8600 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.479187 8600 leader_election.cc:288] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
I20260926 18:02:38.495186 8563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8b9d58914a84061b815691412e4a3f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20" is_pre_election: true
W20260926 18:02:38.495306 8007 leader_election.cc:334] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505): Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:38.495858 8563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8b9d58914a84061b815691412e4a3f2 in term 2.
I20260926 18:02:38.497596 8008 leader_election.cc:302] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, e8b9d58914a84061b815691412e4a3f2; no voters: f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:38.498247 8519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 18:02:38.498580 8519 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f822563edfad492e8f52dc2d8f62d7da)
I20260926 18:02:38.498908 8519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 2 FOLLOWER]: Advancing to term 3
W20260926 18:02:38.499020 8538 leader_election.cc:334] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505): Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:38.504532 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "e8b9d58914a84061b815691412e4a3f2" is_pre_election: true
I20260926 18:02:38.504413 8519 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.505501 8030 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e3ff4442fcb446c0a9eea4effe901f20 in current term 3: Already voted for candidate e8b9d58914a84061b815691412e4a3f2 in this term.
I20260926 18:02:38.506415 8536 leader_election.cc:302] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20; no voters: e8b9d58914a84061b815691412e4a3f2, f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:38.506754 8563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8b9d58914a84061b815691412e4a3f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20"
I20260926 18:02:38.507263 8600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:38.507611 8519 leader_election.cc:288] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 election: Requested vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
W20260926 18:02:38.510213 8007 leader_election.cc:334] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505): Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111)
I20260926 18:02:38.512041 8600 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 18:02:38.515900 8563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8b9d58914a84061b815691412e4a3f2 in term 3.
I20260926 18:02:38.517949 8008 leader_election.cc:302] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, e8b9d58914a84061b815691412e4a3f2; no voters: f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:38.518851 8519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 18:02:38.520030 8519 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 LEADER]: Becoming Leader. State: Replica: e8b9d58914a84061b815691412e4a3f2, State: Running, Role: LEADER
I20260926 18:02:38.520781 8519 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.523377 8620 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8b9d58914a84061b815691412e4a3f2. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.523952 8620 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:38.525246 8621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:38.529412 8621 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e]
I20260926 18:02:38.532928 8621 tablet_loader.cc:96] loaded metadata for tablet 386c7447fb5543c9b6a5f27922017e76 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.533792 8621 tablet_loader.cc:96] loaded metadata for tablet 3f4824a63d564a36a20c87846dbd1c46 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.534660 8621 tablet_loader.cc:96] loaded metadata for tablet ea77d43447de400fa8864da7daab7bca (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:38.535437 8621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:38.538352 8621 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:38.538590 8621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:38.542377 8621 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:38.545288 8621 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Loaded TSK: 0
I20260926 18:02:38.546106 8621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:38.550287 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:38.555440 8622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:38.555785 8623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:38.558511 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:38.558557 8625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:38.559594 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:38.560536 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:38.560726 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445758560710 us; error 0 us; skew 500 ppm
I20260926 18:02:38.561197 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:38.563366 6506 webserver.cc:460] Webserver started at http://127.6.90.188:41187/ using document root <none> and password file <none>
I20260926 18:02:38.563777 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:38.563930 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:38.567958 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 18:02:38.575212 8630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:38.575969 6506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 18:02:38.576238 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
uuid: "f822563edfad492e8f52dc2d8f62d7da"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:38.576516 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:38.602555 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:38.603600 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:38.620950 8563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Refusing update from remote peer e8b9d58914a84061b815691412e4a3f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 18. Preceding OpId from leader: term: 3 index: 19. (index mismatch)
I20260926 18:02:38.622400 8620 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260926 18:02:38.625437 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8b9d58914a84061b815691412e4a3f2. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.626206 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.628773 8620 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.629492 8620 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:38.634795 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.635354 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 18:02:38.636175 8007 consensus_peers.cc:487] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 -> Peer f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505): Couldn't send request to peer f822563edfad492e8f52dc2d8f62d7da. Status: Network error: Client connection negotiation failed: client connection to 127.6.90.188:38505: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:38.644573 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.188:38505
I20260926 18:02:38.644645 8682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.188:38505 every 8 connection(s)
I20260926 18:02:38.652069 8683 sys_catalog.cc:263] Verifying existing consensus state
I20260926 18:02:38.656086 8683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Bootstrap starting.
I20260926 18:02:38.661679 8406 heartbeater.cc:503] Master 127.6.90.189:35189 was elected leader, sending a full tablet report...
I20260926 18:02:38.670290 8328 heartbeater.cc:503] Master 127.6.90.189:35189 was elected leader, sending a full tablet report...
I20260926 18:02:38.672713 8250 heartbeater.cc:503] Master 127.6.90.189:35189 was elected leader, sending a full tablet report...
I20260926 18:02:38.680701 8405 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:38.690655 8327 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:38.697096 8249 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:38.726971 8683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=7 ignored=0} mutations{seen=14 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:38.727695 8683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Bootstrap complete.
I20260926 18:02:38.729677 8683 raft_consensus.cc:348] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.730172 8683 raft_consensus.cc:729] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f822563edfad492e8f52dc2d8f62d7da, State: Initialized, Role: FOLLOWER
I20260926 18:02:38.730700 8683 consensus_queue.cc:260] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.731959 8689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.732527 8689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.733012 8683 sys_catalog.cc:564] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:38.745085 8700 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:38.745316 8700 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: loading cluster ID for follower catalog manager: success
I20260926 18:02:38.749357 6506 master.cc:537] Master@127.6.90.189:35189 shutting down...
I20260926 18:02:38.752431 8647 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" instance_seqno: 1790445755257675) as {username='slave'} at 127.0.0.1:60152; Asking this server to re-register.
I20260926 18:02:38.755308 8405 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.758544 8647 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:38.759763 8700 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: acquiring CA information for follower catalog manager: success
I20260926 18:02:38.759964 8646 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" instance_seqno: 1790445755106376) as {username='slave'} at 127.0.0.1:60170; Asking this server to re-register.
I20260926 18:02:38.762612 8327 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.764389 8646 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" instance_seqno: 1790445754907997) as {username='slave'} at 127.0.0.1:60162; Asking this server to re-register.
I20260926 18:02:38.765245 8647 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:38.765838 8700 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 18:02:38.765970 8249 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:38.774382 8657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 2 FOLLOWER]: Advancing to term 3
I20260926 18:02:38.774461 8647 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:38.784907 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.785640 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.786243 8689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8b9d58914a84061b815691412e4a3f2. Latest consensus state: current_term: 3 leader_uuid: "e8b9d58914a84061b815691412e4a3f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:38.786742 8689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:38.788120 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 LEADER]: Raft consensus shutting down.
I20260926 18:02:38.788968 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:38.789376 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: stopping tablet replica
I20260926 18:02:38.809224 6506 master.cc:552] Master@127.6.90.189:35189 shutdown complete.
I20260926 18:02:38.917078 8600 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8b9d58914a84061b815691412e4a3f2)
I20260926 18:02:38.917469 8600 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.918980 8600 leader_election.cc:288] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189), f822563edfad492e8f52dc2d8f62d7da (127.6.90.188:38505)
W20260926 18:02:38.920424 8536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111) [suppressed 14 similar messages]
W20260926 18:02:38.922564 8536 leader_election.cc:334] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189): Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111)
I20260926 18:02:38.928550 8657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff4442fcb446c0a9eea4effe901f20" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "f822563edfad492e8f52dc2d8f62d7da" is_pre_election: true
I20260926 18:02:38.929613 8538 leader_election.cc:302] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20; no voters: e8b9d58914a84061b815691412e4a3f2, f822563edfad492e8f52dc2d8f62d7da
I20260926 18:02:38.930178 8600 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 18:02:38.981971 8689 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8b9d58914a84061b815691412e4a3f2)
I20260926 18:02:38.982434 8689 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:38.984519 8689 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
W20260926 18:02:38.990686 8631 leader_election.cc:334] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189): Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111)
I20260926 18:02:38.995797 8563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20" is_pre_election: true
I20260926 18:02:38.996294 8563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 3.
I20260926 18:02:38.997047 8634 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, f822563edfad492e8f52dc2d8f62d7da; no voters: e8b9d58914a84061b815691412e4a3f2
I20260926 18:02:38.997655 8689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 18:02:38.997993 8689 raft_consensus.cc:482] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 3 FOLLOWER]: Starting leader election (detected failure of leader e8b9d58914a84061b815691412e4a3f2)
I20260926 18:02:38.998243 8689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:39.002313 8689 raft_consensus.cc:504] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:39.003649 8689 leader_election.cc:288] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 election: Requested vote from peers e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281), e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189)
I20260926 18:02:39.004212 8563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f822563edfad492e8f52dc2d8f62d7da" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "e3ff4442fcb446c0a9eea4effe901f20"
I20260926 18:02:39.004707 8563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 3 FOLLOWER]: Advancing to term 4
W20260926 18:02:39.006552 8631 leader_election.cc:334] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189): Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111)
I20260926 18:02:39.009070 8563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f822563edfad492e8f52dc2d8f62d7da in term 4.
I20260926 18:02:39.009806 8634 leader_election.cc:302] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e3ff4442fcb446c0a9eea4effe901f20, f822563edfad492e8f52dc2d8f62d7da; no voters: e8b9d58914a84061b815691412e4a3f2
I20260926 18:02:39.010380 8689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 4 FOLLOWER]: Leader election won for term 4
I20260926 18:02:39.010711 8689 raft_consensus.cc:686] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 4 LEADER]: Becoming Leader. State: Replica: f822563edfad492e8f52dc2d8f62d7da, State: Running, Role: LEADER
I20260926 18:02:39.011245 8689 consensus_queue.cc:237] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:39.013635 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.014196 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:39.015223 8707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:39.018800 8707 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e]
I20260926 18:02:39.019382 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 18:02:39.022909 8707 tablet_loader.cc:96] loaded metadata for tablet 386c7447fb5543c9b6a5f27922017e76 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
I20260926 18:02:39.024381 8707 tablet_loader.cc:96] loaded metadata for tablet 3f4824a63d564a36a20c87846dbd1c46 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
W20260926 18:02:39.024749 8708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:39.026492 8707 tablet_loader.cc:96] loaded metadata for tablet ea77d43447de400fa8864da7daab7bca (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e])
W20260926 18:02:39.027268 8709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:39.027556 8711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:39.027742 8707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:39.028278 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:39.028945 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:39.029126 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:39.029284 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445759029270 us; error 0 us; skew 500 ppm
I20260926 18:02:39.029739 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:39.031576 8707 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:39.031839 8707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:39.032079 6506 webserver.cc:460] Webserver started at http://127.6.90.189:37511/ using document root <none> and password file <none>
I20260926 18:02:39.032577 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:39.032737 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:39.036051 8707 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:39.036087 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:39.038986 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:39.039752 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:39.040011 8707 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da: Loaded TSK: 0
I20260926 18:02:39.040004 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
uuid: "e8b9d58914a84061b815691412e4a3f2"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:39.040445 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:39.041042 8707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:39.069029 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:39.070163 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:39.103000 8563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 4 FOLLOWER]: Refusing update from remote peer f822563edfad492e8f52dc2d8f62d7da: Log matching property violated. Preceding OpId in replica: term: 3 index: 19. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
I20260926 18:02:39.104223 8703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260926 18:02:39.106505 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.189:35189
I20260926 18:02:39.106604 8767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.189:35189 every 8 connection(s)
I20260926 18:02:39.108620 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.109267 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:39.110381 8689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.111119 8689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:39.113349 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.113940 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:39.119267 8768 sys_catalog.cc:263] Verifying existing consensus state
I20260926 18:02:39.124452 8768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Bootstrap starting.
I20260926 18:02:39.170998 8405 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:39.175880 8327 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:39.190658 8249 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:39.195852 8768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=7 ignored=0} mutations{seen=14 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:39.196805 8768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: Bootstrap complete.
I20260926 18:02:39.199501 8768 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:39.200206 8768 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b9d58914a84061b815691412e4a3f2, State: Initialized, Role: FOLLOWER
I20260926 18:02:39.200999 8768 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } }
I20260926 18:02:39.203559 8775 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.204279 8775 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:39.204866 8768 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:39.205113 8743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:39.210283 8743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 4 FOLLOWER]: Refusing update from remote peer f822563edfad492e8f52dc2d8f62d7da: Log matching property violated. Preceding OpId in replica: term: 3 index: 19. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
I20260926 18:02:39.254608 8703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260926 18:02:39.264739 8328 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:39.268424 8250 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:39.278226 8406 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:39.293556 8789 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.294159 8775 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f822563edfad492e8f52dc2d8f62d7da. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.294621 8789 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:39.294971 8775 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 18:02:39.299618 8790 catalog_manager.cc:1259] Loaded cluster ID: 9662484e52c94572b702c5d702baa2de
I20260926 18:02:39.299823 8790 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: loading cluster ID for follower catalog manager: success
I20260926 18:02:39.304822 8790 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: acquiring CA information for follower catalog manager: success
I20260926 18:02:39.308462 8790 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 18:02:39.317335 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:39.336926 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:39.337822 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:39.341990 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:39.342636 6506 tablet_replica.cc:330] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: stopping tablet replica
I20260926 18:02:39.343184 6506 raft_consensus.cc:2229] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 LEADER]: Raft consensus shutting down.
I20260926 18:02:39.344101 6506 raft_consensus.cc:2258] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:39.346962 6506 tablet_replica.cc:330] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: stopping tablet replica
I20260926 18:02:39.347436 6506 raft_consensus.cc:2229] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:39.347899 6506 raft_consensus.cc:2258] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260926 18:02:39.349996 8266 consensus_peers.cc:487] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:39.350543 6506 tablet_replica.cc:330] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: stopping tablet replica
I20260926 18:02:39.350982 6506 raft_consensus.cc:2229] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:39.351569 6506 raft_consensus.cc:2258] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260926 18:02:39.367498 8266 consensus_peers.cc:487] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Status: Network error: Client connection negotiation failed: client connection to 127.6.90.129:41673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:39.374017 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:39.378358 8732 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" instance_seqno: 1790445755106376) as {username='slave'} at 127.0.0.1:45266; Asking this server to re-register.
I20260926 18:02:39.379598 8328 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:39.383005 8732 ts_manager.cc:194] Registered new tserver with Master: 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:39.391943 8732 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" instance_seqno: 1790445755257675) as {username='slave'} at 127.0.0.1:45276; Asking this server to re-register.
I20260926 18:02:39.393209 8406 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:39.393577 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 18:02:39.396050 8732 ts_manager.cc:194] Registered new tserver with Master: 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
W20260926 18:02:39.400822 8792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:39.400991 8793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:39.402969 8795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:39.403941 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:39.405591 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:39.405781 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:39.405942 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445759405928 us; error 0 us; skew 500 ppm
I20260926 18:02:39.406456 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:39.408867 6506 webserver.cc:460] Webserver started at http://127.6.90.129:35183/ using document root <none> and password file <none>
I20260926 18:02:39.409307 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:39.409467 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:39.412953 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 18:02:39.415504 8800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:39.416177 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 18:02:39.416453 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "9a0e1006c4d94358929b16cbf0f4a32f"
format_stamp: "Formatted at 2026-09-26 18:02:34 on dist-test-slave-5fqf"
I20260926 18:02:39.416770 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:39.431066 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:39.432171 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:39.433290 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:39.436326 8807 ts_tablet_manager.cc:542] Loading tablet metadata (0/3 complete)
I20260926 18:02:39.451296 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260926 18:02:39.451563 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260926 18:02:39.451856 6506 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20260926 18:02:39.456252 8807 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:39.462764 6506 ts_tablet_manager.cc:610] Registered 3 tablets
I20260926 18:02:39.463018 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.011s user 0.006s sys 0.003s
I20260926 18:02:39.515085 8846 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:39.515730 8846 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.518102 8846 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130:38067)
I20260926 18:02:39.520879 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "67a96471cc6b46e3ad4324475a2e472f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" is_pre_election: true
W20260926 18:02:39.533401 8344 leader_election.cc:334] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Network error: Client connection negotiation failed: client connection to 127.6.90.129:41673: connect: Connection refused (error 111)
I20260926 18:02:39.533955 8344 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67a96471cc6b46e3ad4324475a2e472f; no voters: 07f5d1825d624bd1ad7bc65fe13637b6, 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:39.534902 8846 raft_consensus.cc:2726] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 18:02:39.539954 8513 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:39.540618 8513 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.543000 8513 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:39.545254 8807 tablet.cc:1722] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:39.546113 8807 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260926 18:02:39.546763 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f" is_pre_election: true
I20260926 18:02:39.547305 8807 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=260 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:39.547441 8380 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 3.
I20260926 18:02:39.548202 8807 tablet_bootstrap.cc:492] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap complete.
I20260926 18:02:39.549014 8267 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters:
W20260926 18:02:39.550242 8266 leader_election.cc:334] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Network error: Client connection negotiation failed: client connection to 127.6.90.129:41673: connect: Connection refused (error 111)
I20260926 18:02:39.550855 8810 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 18:02:39.551288 8810 raft_consensus.cc:482] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9a0e1006c4d94358929b16cbf0f4a32f)
I20260926 18:02:39.551657 8810 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:39.559968 8810 raft_consensus.cc:504] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.562309 8807 ts_tablet_manager.cc:1397] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.106s user 0.078s sys 0.016s
I20260926 18:02:39.563196 8380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4824a63d564a36a20c87846dbd1c46" candidate_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "67a96471cc6b46e3ad4324475a2e472f"
I20260926 18:02:39.563799 8380 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:39.564803 8807 raft_consensus.cc:348] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:39.565514 8807 raft_consensus.cc:729] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Initialized, Role: FOLLOWER
W20260926 18:02:39.566015 8266 leader_election.cc:334] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Network error: Client connection negotiation failed: client connection to 127.6.90.129:41673: connect: Connection refused (error 111)
I20260926 18:02:39.566120 8807 consensus_queue.cc:260] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } }
I20260926 18:02:39.568723 8807 ts_tablet_manager.cc:1428] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 18:02:39.569377 8807 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:39.570535 8380 raft_consensus.cc:2454] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5d1825d624bd1ad7bc65fe13637b6 in term 4.
I20260926 18:02:39.571523 8869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:41673 every 8 connection(s)
I20260926 18:02:39.572098 8810 leader_election.cc:288] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 election: Requested vote from peers 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673), 67a96471cc6b46e3ad4324475a2e472f (127.6.90.131:39391)
I20260926 18:02:39.571383 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:41673
I20260926 18:02:39.573135 8267 leader_election.cc:302] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f5d1825d624bd1ad7bc65fe13637b6, 67a96471cc6b46e3ad4324475a2e472f; no voters: 9a0e1006c4d94358929b16cbf0f4a32f
I20260926 18:02:39.574229 8810 raft_consensus.cc:2781] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 18:02:39.576938 8810 raft_consensus.cc:686] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 LEADER]: Becoming Leader. State: Replica: 07f5d1825d624bd1ad7bc65fe13637b6, State: Running, Role: LEADER
I20260926 18:02:39.577893 8810 consensus_queue.cc:237] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.651587 8646 catalog_manager.cc:5495] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 reported cstate change: term changed from 3 to 4, leader changed from 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) to 07f5d1825d624bd1ad7bc65fe13637b6 (127.6.90.130). New cstate: current_term: 4 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:39.667826 8871 heartbeater.cc:347] Connected to a master server at 127.6.90.188:38505
I20260926 18:02:39.668262 8871 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:39.669153 8871 heartbeater.cc:511] Master 127.6.90.188:38505 requested a full tablet report, sending...
I20260926 18:02:39.673440 8647 ts_manager.cc:194] Re-registered known tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
W20260926 18:02:39.693131 8266 consensus_peers.cc:487] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:39.694239 8874 heartbeater.cc:347] Connected to a master server at 127.6.90.189:35189
I20260926 18:02:39.694617 8874 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:39.695462 8874 heartbeater.cc:511] Master 127.6.90.189:35189 requested a full tablet report, sending...
I20260926 18:02:39.699914 8733 ts_manager.cc:194] Registered new tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:39.703078 8647 catalog_manager.cc:5587] TS 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673) does not have the latest schema for tablet 3f4824a63d564a36a20c87846dbd1c46 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e]). Expected version 1 got 0
I20260926 18:02:39.707695 8380 raft_consensus.cc:1261] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260926 18:02:39.709353 8513 consensus_queue.cc:1035] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 18:02:39.712662 8647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60188
I20260926 18:02:39.717417 8871 heartbeater.cc:503] Master 127.6.90.188:38505 was elected leader, sending a full tablet report...
I20260926 18:02:39.740824 8647 catalog_manager.cc:5587] TS 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673) does not have the latest schema for tablet 3f4824a63d564a36a20c87846dbd1c46 (table NewTableName [id=399225e6572b4962b4ccc60d8e35e29e]). Expected version 1 got 0
I20260926 18:02:39.746170 8807 tablet.cc:1722] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:39.746860 8807 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260926 18:02:39.747895 8807 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=266 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:39.748709 8807 tablet_bootstrap.cc:492] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap complete.
I20260926 18:02:39.758772 8877 heartbeater.cc:347] Connected to a master server at 127.6.90.190:40281
I20260926 18:02:39.759187 8877 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:39.759884 8877 heartbeater.cc:511] Master 127.6.90.190:40281 requested a full tablet report, sending...
I20260926 18:02:39.766726 8553 ts_manager.cc:194] Re-registered known tserver with Master: 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673)
I20260926 18:02:39.775107 8807 ts_tablet_manager.cc:1397] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.206s user 0.094s sys 0.052s
I20260926 18:02:39.776681 8807 raft_consensus.cc:348] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.777143 8807 raft_consensus.cc:729] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0e1006c4d94358929b16cbf0f4a32f, State: Initialized, Role: FOLLOWER
I20260926 18:02:39.777582 8807 consensus_queue.cc:260] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 41673 } } peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.778834 8807 ts_tablet_manager.cc:1428] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:39.779668 8807 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap starting.
I20260926 18:02:39.790343 8841 raft_consensus.cc:3037] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 3 FOLLOWER]: Advancing to term 4
I20260926 18:02:39.835812 8807 tablet.cc:1722] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 18:02:39.836324 8807 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260926 18:02:39.837100 8807 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=276 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:39.837801 8807 tablet_bootstrap.cc:492] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Bootstrap complete.
I20260926 18:02:39.843698 8807 ts_tablet_manager.cc:1397] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent bootstrapping tablet: real 0.064s user 0.047s sys 0.012s
E20260926 18:02:39.845248 8807 ts_tablet_manager.cc:1439] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Tablet failed to start: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260926 18:02:39.845494 8807 ts_tablet_manager.cc:1428] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260926 18:02:39.845671 8807 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:39.845813 8807 raft_consensus.cc:2229] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:39.846041 8807 raft_consensus.cc:2258] T ea77d43447de400fa8864da7daab7bca P 9a0e1006c4d94358929b16cbf0f4a32f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:39.855063 8266 consensus_queue.cc:579] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: The tablet replica hosted on peer 9a0e1006c4d94358929b16cbf0f4a32f has failed
W20260926 18:02:39.855351 8266 consensus_peers.cc:487] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2. This is attempt 6: this message will repeat every 5th retry.
I20260926 18:02:39.862875 8301 consensus_queue.cc:237] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } }
I20260926 18:02:39.867003 8380 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260926 18:02:39.868021 8502 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 18:02:39.874056 8502 raft_consensus.cc:2932] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } }
I20260926 18:02:39.880630 8380 raft_consensus.cc:2932] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index -1 to 5, VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } }
I20260926 18:02:39.887262 8631 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet ea77d43447de400fa8864da7daab7bca with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260926 18:02:39.895502 8647 catalog_manager.cc:5495] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f reported cstate change: config changed from index -1 to 5, VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) evicted. New cstate: current_term: 3 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } }
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:906: Failure
Failed
Bad status: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
I20260926 18:02:39.904404 6506 tablet_server.cc:178] TabletServer@127.6.90.129:41673 shutting down...
I20260926 18:02:39.928833 8301 consensus_queue.cc:237] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 41673 } attrs { promote: true } }
W20260926 18:02:39.935102 8266 consensus_peers.cc:487] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:39.935526 8380 raft_consensus.cc:1261] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Refusing update from remote peer 07f5d1825d624bd1ad7bc65fe13637b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 18:02:39.936738 8810 consensus_queue.cc:1035] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260926 18:02:39.943236 8632 catalog_manager.cc:4584] TS 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ea77d43447de400fa8864da7daab7bca: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260926 18:02:39.943903 8513 raft_consensus.cc:2932] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 41673 } attrs { promote: true } } }
I20260926 18:02:39.946746 8380 raft_consensus.cc:2932] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, NON_VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 41673 } attrs { promote: true } } }
I20260926 18:02:39.954435 8631 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ea77d43447de400fa8864da7daab7bca with cas_config_opid_index 5: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260926 18:02:39.962760 8646 catalog_manager.cc:5495] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f reported cstate change: config changed from index 5 to 6, NON_VOTER 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129) added. New cstate: current_term: 3 leader_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "07f5d1825d624bd1ad7bc65fe13637b6" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 38067 } } peers { permanent_uuid: "67a96471cc6b46e3ad4324475a2e472f" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 39391 } } peers { permanent_uuid: "9a0e1006c4d94358929b16cbf0f4a32f" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 41673 } attrs { promote: true } } }
W20260926 18:02:39.964634 8266 consensus_peers.cc:487] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:39.972178 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:39.976064 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:39.976773 6506 raft_consensus.cc:2229] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:39.977260 6506 raft_consensus.cc:2258] T 3f4824a63d564a36a20c87846dbd1c46 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:39.983673 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:39.984210 6506 raft_consensus.cc:2229] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:39.984653 6506 raft_consensus.cc:2258] T 386c7447fb5543c9b6a5f27922017e76 P 9a0e1006c4d94358929b16cbf0f4a32f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260926 18:02:39.986490 8266 consensus_peers.cc:487] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 -> Peer 9a0e1006c4d94358929b16cbf0f4a32f (127.6.90.129:41673): Couldn't send request to peer 9a0e1006c4d94358929b16cbf0f4a32f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:40.005959 6506 tablet_server.cc:194] TabletServer@127.6.90.129:41673 shutdown complete.
I20260926 18:02:40.014731 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:40.031538 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:40.033440 6506 tablet_replica.cc:330] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6: stopping tablet replica
I20260926 18:02:40.034140 6506 raft_consensus.cc:2229] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 LEADER]: Raft consensus shutting down.
W20260926 18:02:40.034607 8513 consensus_queue.cc:1159] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "67a96471cc6b46e3ad4324475a2e472f" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260926 18:02:40.035383 6506 raft_consensus.cc:2258] T 3f4824a63d564a36a20c87846dbd1c46 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.038058 6506 tablet_replica.cc:330] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6: stopping tablet replica
I20260926 18:02:40.038516 6506 raft_consensus.cc:2229] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 LEADER]: Raft consensus shutting down.
I20260926 18:02:40.039155 6506 raft_consensus.cc:2258] T ea77d43447de400fa8864da7daab7bca P 07f5d1825d624bd1ad7bc65fe13637b6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.041392 6506 tablet_replica.cc:330] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6: stopping tablet replica
I20260926 18:02:40.041828 6506 raft_consensus.cc:2229] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 LEADER]: Raft consensus shutting down.
I20260926 18:02:40.042451 6506 raft_consensus.cc:2258] T 386c7447fb5543c9b6a5f27922017e76 P 07f5d1825d624bd1ad7bc65fe13637b6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.063460 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:40.074860 6506 tablet_server.cc:178] TabletServer@127.6.90.131:0 shutting down...
I20260926 18:02:40.098380 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:40.098961 6506 tablet_replica.cc:330] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f: stopping tablet replica
I20260926 18:02:40.099462 6506 raft_consensus.cc:2229] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:40.099864 6506 raft_consensus.cc:2258] T 3f4824a63d564a36a20c87846dbd1c46 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.102121 6506 tablet_replica.cc:330] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f: stopping tablet replica
I20260926 18:02:40.102542 6506 raft_consensus.cc:2229] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:40.102936 6506 raft_consensus.cc:2258] T ea77d43447de400fa8864da7daab7bca P 67a96471cc6b46e3ad4324475a2e472f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.105304 6506 tablet_replica.cc:330] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f: stopping tablet replica
I20260926 18:02:40.105700 6506 raft_consensus.cc:2229] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:40.106088 6506 raft_consensus.cc:2258] T 386c7447fb5543c9b6a5f27922017e76 P 67a96471cc6b46e3ad4324475a2e472f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.126139 6506 tablet_server.cc:194] TabletServer@127.6.90.131:0 shutdown complete.
I20260926 18:02:40.140069 6506 master.cc:537] Master@127.6.90.190:40281 shutting down...
I20260926 18:02:40.156540 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:40.156970 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e3ff4442fcb446c0a9eea4effe901f20 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.157269 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:40.175473 6506 master.cc:552] Master@127.6.90.190:40281 shutdown complete.
W20260926 18:02:40.187955 8634 consensus_peers.cc:487] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da -> Peer e3ff4442fcb446c0a9eea4effe901f20 (127.6.90.190:40281): Couldn't send request to peer e3ff4442fcb446c0a9eea4effe901f20. Status: Network error: Client connection negotiation failed: client connection to 127.6.90.190:40281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:40.188786 6506 master.cc:537] Master@127.6.90.189:35189 shutting down...
I20260926 18:02:40.202947 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:40.203400 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e8b9d58914a84061b815691412e4a3f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.203675 6506 tablet_replica.cc:330] stopping tablet replica
W20260926 18:02:40.207501 8718 connection.cc:441] server connection from 127.0.0.1:45244 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 15) could send its response
W20260926 18:02:40.210198 8631 consensus_peers.cc:487] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da -> Peer e8b9d58914a84061b815691412e4a3f2 (127.6.90.189:35189): Couldn't send request to peer e8b9d58914a84061b815691412e4a3f2. Status: Network error: Client connection negotiation failed: client connection to 127.6.90.189:35189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:40.212961 6506 master.cc:552] Master@127.6.90.189:35189 shutdown complete.
I20260926 18:02:40.223688 6506 master.cc:537] Master@127.6.90.188:38505 shutting down...
I20260926 18:02:40.290524 8689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:40.291262 8689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:40.316510 8898 sys_catalog.cc:455] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "f822563edfad492e8f52dc2d8f62d7da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff4442fcb446c0a9eea4effe901f20" member_type: VOTER last_known_addr { host: "127.6.90.190" port: 40281 } } peers { permanent_uuid: "e8b9d58914a84061b815691412e4a3f2" member_type: VOTER last_known_addr { host: "127.6.90.189" port: 35189 } } peers { permanent_uuid: "f822563edfad492e8f52dc2d8f62d7da" member_type: VOTER last_known_addr { host: "127.6.90.188" port: 38505 } } }
I20260926 18:02:40.317219 8898 sys_catalog.cc:458] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:40.321106 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 4 LEADER]: Raft consensus shutting down.
I20260926 18:02:40.321777 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f822563edfad492e8f52dc2d8f62d7da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:40.322148 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:40.339579 6506 master.cc:552] Master@127.6.90.188:38505 shutdown complete.
I20260926 18:02:40.357087 6506 test_util.cc:175] -----------------------------------------------
I20260926 18:02:40.357254 6506 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1790445738975945-6506-0
[ FAILED ] TsTabletManagerITest.TestTableStats (5932 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260926 18:02:40.362918 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:39059
I20260926 18:02:40.363889 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:40.369009 8904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.369932 8905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.372416 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:40.372637 8907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.373409 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:40.373585 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:40.373721 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445760373708 us; error 0 us; skew 500 ppm
I20260926 18:02:40.374198 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:40.376292 6506 webserver.cc:460] Webserver started at http://127.6.90.190:42631/ using document root <none> and password file <none>
I20260926 18:02:40.376745 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:40.376924 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:40.377174 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:40.378180 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "a34856139cf14318ab378e7de2e66ef2"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.382118 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 18:02:40.385003 8912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.385634 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260926 18:02:40.385865 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "a34856139cf14318ab378e7de2e66ef2"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.386114 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:40.401050 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:40.402032 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:40.435300 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:39059
I20260926 18:02:40.435384 8963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:39059 every 8 connection(s)
I20260926 18:02:40.438676 8964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:40.447431 8964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: Bootstrap starting.
I20260926 18:02:40.451305 8964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:40.454798 8964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: No bootstrap required, opened a new log
I20260926 18:02:40.456548 8964 raft_consensus.cc:348] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER }
I20260926 18:02:40.456887 8964 raft_consensus.cc:374] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:40.457082 8964 raft_consensus.cc:729] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34856139cf14318ab378e7de2e66ef2, State: Initialized, Role: FOLLOWER
I20260926 18:02:40.457568 8964 consensus_queue.cc:260] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER }
I20260926 18:02:40.457962 8964 raft_consensus.cc:388] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:40.458161 8964 raft_consensus.cc:482] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:40.458386 8964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:40.462208 8964 raft_consensus.cc:504] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER }
I20260926 18:02:40.462646 8964 leader_election.cc:302] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a34856139cf14318ab378e7de2e66ef2; no voters:
I20260926 18:02:40.463573 8964 leader_election.cc:288] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:40.463827 8967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:40.464998 8967 raft_consensus.cc:686] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 1 LEADER]: Becoming Leader. State: Replica: a34856139cf14318ab378e7de2e66ef2, State: Running, Role: LEADER
I20260926 18:02:40.465567 8967 consensus_queue.cc:237] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER }
I20260926 18:02:40.466117 8964 sys_catalog.cc:564] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:40.468147 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a34856139cf14318ab378e7de2e66ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER } }
I20260926 18:02:40.468204 8969 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a34856139cf14318ab378e7de2e66ef2. Latest consensus state: current_term: 1 leader_uuid: "a34856139cf14318ab378e7de2e66ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34856139cf14318ab378e7de2e66ef2" member_type: VOTER } }
I20260926 18:02:40.469012 8969 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:40.469009 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:40.471316 8973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:40.477922 8973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:40.478873 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:40.485324 8973 catalog_manager.cc:1347] Generated new cluster ID: cb82efaaf1f048fca8598f9e5064e5a8
I20260926 18:02:40.485515 8973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:40.512468 8973 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:40.513983 8973 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:40.530530 8973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: Generated new TSK 0
I20260926 18:02:40.531126 8973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:40.544857 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:40.550424 8986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.550498 8985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.552868 8988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.552935 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:40.553851 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:40.554023 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:40.554167 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445760554152 us; error 0 us; skew 500 ppm
I20260926 18:02:40.554633 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:40.556674 6506 webserver.cc:460] Webserver started at http://127.6.90.129:42275/ using document root <none> and password file <none>
I20260926 18:02:40.557096 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:40.557257 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:40.557471 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:40.558420 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "a0c99ae6bb45484e80360b1bb738a830"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.562420 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 18:02:40.565323 8993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.566025 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:40.566262 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "a0c99ae6bb45484e80360b1bb738a830"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.566507 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:40.580686 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:40.581558 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:40.582805 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:40.584791 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:40.584967 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.585163 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:40.585294 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.620971 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:36913
I20260926 18:02:40.621059 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:36913 every 8 connection(s)
I20260926 18:02:40.624959 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:40.631610 9060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.633311 9061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.635794 9063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.637298 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:40.638120 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:40.638315 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:40.638500 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445760638484 us; error 0 us; skew 500 ppm
I20260926 18:02:40.638652 9056 heartbeater.cc:347] Connected to a master server at 127.6.90.190:39059
I20260926 18:02:40.638968 9056 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:40.639038 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:40.639583 9056 heartbeater.cc:511] Master 127.6.90.190:39059 requested a full tablet report, sending...
I20260926 18:02:40.641557 6506 webserver.cc:460] Webserver started at http://127.6.90.130:33943/ using document root <none> and password file <none>
I20260926 18:02:40.641615 8929 ts_manager.cc:194] Registered new tserver with Master: a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913)
I20260926 18:02:40.642087 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:40.642303 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:40.642526 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:40.643467 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "0d61f688567c4bafab9be7b985ce6f4c"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.643548 8929 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47984
I20260926 18:02:40.647434 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 18:02:40.650417 9068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.651124 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 18:02:40.651347 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "0d61f688567c4bafab9be7b985ce6f4c"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.651575 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:40.664893 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:40.665824 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:40.667258 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:40.669325 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:40.669507 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.669714 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:40.669842 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.704978 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:35029
I20260926 18:02:40.705056 9130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:35029 every 8 connection(s)
I20260926 18:02:40.709127 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:40.715444 9134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:40.717406 9135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.720140 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:40.720577 9137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:40.721480 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:40.721681 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:40.721863 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445760721849 us; error 0 us; skew 500 ppm
I20260926 18:02:40.722311 9131 heartbeater.cc:347] Connected to a master server at 127.6.90.190:39059
I20260926 18:02:40.722457 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:40.722635 9131 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:40.723371 9131 heartbeater.cc:511] Master 127.6.90.190:39059 requested a full tablet report, sending...
I20260926 18:02:40.725157 6506 webserver.cc:460] Webserver started at http://127.6.90.131:43135/ using document root <none> and password file <none>
I20260926 18:02:40.725437 8929 ts_manager.cc:194] Registered new tserver with Master: 0d61f688567c4bafab9be7b985ce6f4c (127.6.90.130:35029)
I20260926 18:02:40.725749 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:40.725988 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:40.726284 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:40.727463 8929 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47998
I20260926 18:02:40.727564 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root/instance:
uuid: "61c50e5c943d40978ddd2644ca77602d"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.731840 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 18:02:40.734740 9142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.735492 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:40.735721 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root
uuid: "61c50e5c943d40978ddd2644ca77602d"
format_stamp: "Formatted at 2026-09-26 18:02:40 on dist-test-slave-5fqf"
I20260926 18:02:40.735939 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:40.759328 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:40.760401 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:40.761709 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:40.763638 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:40.763800 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.764015 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:40.764153 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:40.800274 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.131:42083
I20260926 18:02:40.800387 9204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.131:42083 every 8 connection(s)
I20260926 18:02:40.813730 9205 heartbeater.cc:347] Connected to a master server at 127.6.90.190:39059
I20260926 18:02:40.814136 9205 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:40.814772 9205 heartbeater.cc:511] Master 127.6.90.190:39059 requested a full tablet report, sending...
I20260926 18:02:40.816532 8929 ts_manager.cc:194] Registered new tserver with Master: 61c50e5c943d40978ddd2644ca77602d (127.6.90.131:42083)
I20260926 18:02:40.817597 6506 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014522621s
I20260926 18:02:40.818178 8929 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48014
I20260926 18:02:40.850512 8929 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260926 18:02:40.852164 8929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 18:02:40.898623 9170 tablet_service.cc:1467] Processing CreateTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 (DEFAULT_TABLE table=test-table [id=12e000e2814b4d4cafa847143fe8579a]), partition=
I20260926 18:02:40.899664 9170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077c137bf0f945b7a9b3b0876c3009a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:40.904292 9021 tablet_service.cc:1467] Processing CreateTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 (DEFAULT_TABLE table=test-table [id=12e000e2814b4d4cafa847143fe8579a]), partition=
I20260926 18:02:40.905103 9021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077c137bf0f945b7a9b3b0876c3009a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:40.907955 9096 tablet_service.cc:1467] Processing CreateTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 (DEFAULT_TABLE table=test-table [id=12e000e2814b4d4cafa847143fe8579a]), partition=
I20260926 18:02:40.908926 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077c137bf0f945b7a9b3b0876c3009a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:40.914095 9221 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Bootstrap starting.
I20260926 18:02:40.918087 9222 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Bootstrap starting.
I20260926 18:02:40.920478 9221 tablet_bootstrap.cc:654] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:40.921568 9223 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Bootstrap starting.
I20260926 18:02:40.923295 9222 tablet_bootstrap.cc:654] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:40.927217 9223 tablet_bootstrap.cc:654] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:40.929421 9222 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: No bootstrap required, opened a new log
I20260926 18:02:40.929790 9222 ts_tablet_manager.cc:1397] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260926 18:02:40.931862 9221 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: No bootstrap required, opened a new log
I20260926 18:02:40.932288 9221 ts_tablet_manager.cc:1397] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.011s
I20260926 18:02:40.932407 9222 raft_consensus.cc:348] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:40.933175 9222 raft_consensus.cc:374] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:40.933447 9222 raft_consensus.cc:729] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c99ae6bb45484e80360b1bb738a830, State: Initialized, Role: FOLLOWER
I20260926 18:02:40.934077 9222 consensus_queue.cc:260] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:40.934824 9221 raft_consensus.cc:348] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:40.935586 9221 raft_consensus.cc:374] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:40.935900 9221 raft_consensus.cc:729] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c50e5c943d40978ddd2644ca77602d, State: Initialized, Role: FOLLOWER
I20260926 18:02:40.936532 9056 heartbeater.cc:503] Master 127.6.90.190:39059 was elected leader, sending a full tablet report...
I20260926 18:02:40.937036 9222 ts_tablet_manager.cc:1428] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 18:02:40.936573 9221 consensus_queue.cc:260] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
W20260926 18:02:40.937366 9222 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260926 18:02:40.939000 9205 heartbeater.cc:503] Master 127.6.90.190:39059 was elected leader, sending a full tablet report...
I20260926 18:02:40.939296 9223 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: No bootstrap required, opened a new log
I20260926 18:02:40.939409 9221 ts_tablet_manager.cc:1428] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260926 18:02:40.939667 9221 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260926 18:02:40.939752 9223 ts_tablet_manager.cc:1397] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260926 18:02:40.941929 9223 raft_consensus.cc:348] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:40.942412 9223 raft_consensus.cc:374] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:40.942708 9223 raft_consensus.cc:729] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d61f688567c4bafab9be7b985ce6f4c, State: Initialized, Role: FOLLOWER
I20260926 18:02:40.943349 9223 consensus_queue.cc:260] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:40.945392 9131 heartbeater.cc:503] Master 127.6.90.190:39059 was elected leader, sending a full tablet report...
I20260926 18:02:40.945672 9223 ts_tablet_manager.cc:1428] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
W20260926 18:02:40.945886 9223 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260926 18:02:41.149263 9228 raft_consensus.cc:482] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:41.149734 9228 raft_consensus.cc:504] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:41.151494 9228 leader_election.cc:288] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913), 0d61f688567c4bafab9be7b985ce6f4c (127.6.90.130:35029)
I20260926 18:02:41.161906 9031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077c137bf0f945b7a9b3b0876c3009a5" candidate_uuid: "61c50e5c943d40978ddd2644ca77602d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c99ae6bb45484e80360b1bb738a830" is_pre_election: true
I20260926 18:02:41.162423 9031 raft_consensus.cc:2454] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61c50e5c943d40978ddd2644ca77602d in term 0.
I20260926 18:02:41.163280 9146 leader_election.cc:302] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c50e5c943d40978ddd2644ca77602d, a0c99ae6bb45484e80360b1bb738a830; no voters:
I20260926 18:02:41.164072 9228 raft_consensus.cc:2781] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:41.164456 9228 raft_consensus.cc:482] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:41.164788 9228 raft_consensus.cc:3037] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:41.170583 9106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077c137bf0f945b7a9b3b0876c3009a5" candidate_uuid: "61c50e5c943d40978ddd2644ca77602d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d61f688567c4bafab9be7b985ce6f4c" is_pre_election: true
I20260926 18:02:41.170414 9228 raft_consensus.cc:504] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:41.171211 9106 raft_consensus.cc:2454] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61c50e5c943d40978ddd2644ca77602d in term 0.
I20260926 18:02:41.172214 9228 leader_election.cc:288] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [CANDIDATE]: Term 1 election: Requested vote from peers a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913), 0d61f688567c4bafab9be7b985ce6f4c (127.6.90.130:35029)
I20260926 18:02:41.172816 9031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077c137bf0f945b7a9b3b0876c3009a5" candidate_uuid: "61c50e5c943d40978ddd2644ca77602d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c99ae6bb45484e80360b1bb738a830"
I20260926 18:02:41.173245 9031 raft_consensus.cc:3037] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:41.173454 9106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077c137bf0f945b7a9b3b0876c3009a5" candidate_uuid: "61c50e5c943d40978ddd2644ca77602d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d61f688567c4bafab9be7b985ce6f4c"
I20260926 18:02:41.173890 9106 raft_consensus.cc:3037] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:41.177250 9031 raft_consensus.cc:2454] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61c50e5c943d40978ddd2644ca77602d in term 1.
I20260926 18:02:41.177977 9106 raft_consensus.cc:2454] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61c50e5c943d40978ddd2644ca77602d in term 1.
I20260926 18:02:41.178066 9146 leader_election.cc:302] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c50e5c943d40978ddd2644ca77602d, a0c99ae6bb45484e80360b1bb738a830; no voters:
I20260926 18:02:41.179401 9228 raft_consensus.cc:2781] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:41.180178 9228 raft_consensus.cc:686] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 LEADER]: Becoming Leader. State: Replica: 61c50e5c943d40978ddd2644ca77602d, State: Running, Role: LEADER
I20260926 18:02:41.180861 9228 consensus_queue.cc:237] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:41.186640 8929 catalog_manager.cc:5495] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d reported cstate change: term changed from 0 to 1, leader changed from <none> to 61c50e5c943d40978ddd2644ca77602d (127.6.90.131). New cstate: current_term: 1 leader_uuid: "61c50e5c943d40978ddd2644ca77602d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:41.286749 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:42834
I20260926 18:02:41.288090 9234 tablet_replica.cc:330] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: stopping tablet replica
I20260926 18:02:41.288667 9234 raft_consensus.cc:2229] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:41.289057 9234 raft_consensus.cc:2258] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:41.290784 9234 ts_tablet_manager.cc:1905] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260926 18:02:41.299572 9234 ts_tablet_manager.cc:1918] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260926 18:02:41.299853 9234 log.cc:1196] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root/wals/077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.303076 9235 ts_tablet_manager.cc:927] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Initiating tablet copy from peer 61c50e5c943d40978ddd2644ca77602d (127.6.90.131:42083)
I20260926 18:02:41.304558 9235 tablet_copy_client.cc:316] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet copy: Beginning tablet copy session from remote peer at address 127.6.90.131:42083
I20260926 18:02:41.314024 9190 tablet_copy_service.cc:140] P 61c50e5c943d40978ddd2644ca77602d: Received BeginTabletCopySession request for tablet 077c137bf0f945b7a9b3b0876c3009a5 from peer a0c99ae6bb45484e80360b1bb738a830 ({username='slave'} at 127.0.0.1:58612)
I20260926 18:02:41.314460 9190 tablet_copy_service.cc:161] P 61c50e5c943d40978ddd2644ca77602d: Beginning new tablet copy session on tablet 077c137bf0f945b7a9b3b0876c3009a5 from peer a0c99ae6bb45484e80360b1bb738a830 at {username='slave'} at 127.0.0.1:58612: session id = a0c99ae6bb45484e80360b1bb738a830-077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.319182 9190 tablet_copy_source_session.cc:205] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 18:02:41.321290 9235 ts_tablet_manager.cc:1905] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260926 18:02:41.329947 9235 ts_tablet_manager.cc:1918] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260926 18:02:41.330439 9235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077c137bf0f945b7a9b3b0876c3009a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:41.334642 9235 tablet_copy_client.cc:791] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet copy: Starting download of 0 data blocks...
I20260926 18:02:41.335023 9235 tablet_copy_client.cc:655] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet copy: Starting download of 1 WAL segments...
I20260926 18:02:41.336529 8929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48034:
table { table_id: "12e000e2814b4d4cafa847143fe8579a" table_name: "test-table" }
I20260926 18:02:41.337159 8929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48034:
table { table_id: "12e000e2814b4d4cafa847143fe8579a" table_name: "test-table" }
I20260926 18:02:41.338672 9235 tablet_copy_client.cc:531] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 18:02:41.343477 9235 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Bootstrap starting.
I20260926 18:02:41.345773 8929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: Sending DeleteTablet for 3 replicas of tablet 077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.347838 9170 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 18:02:41 UTC) from {username='slave'} at 127.0.0.1:58590
I20260926 18:02:41.347820 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 18:02:41 UTC) from {username='slave'} at 127.0.0.1:42802
I20260926 18:02:41.349545 9096 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 18:02:41 UTC) from {username='slave'} at 127.0.0.1:50814
I20260926 18:02:41.350371 9238 tablet_replica.cc:330] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: stopping tablet replica
W20260926 18:02:41.350483 8916 catalog_manager.cc:4821] TS a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913): delete failed for tablet 077c137bf0f945b7a9b3b0876c3009a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 077c137bf0f945b7a9b3b0876c3009a5 already in progress: copying tablet
I20260926 18:02:41.351105 9239 tablet_replica.cc:330] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: stopping tablet replica
I20260926 18:02:41.351275 9238 raft_consensus.cc:2229] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:41.351821 9239 raft_consensus.cc:2229] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:41.352178 9238 pending_rounds.cc:62] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Trying to abort 1 pending ops.
I20260926 18:02:41.352272 9239 raft_consensus.cc:2258] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:41.352718 9238 pending_rounds.cc:68] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333665837803347968 op_type: NO_OP noop_request { }
I20260926 18:02:41.353324 9238 raft_consensus.cc:2866] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 18:02:41.353668 9238 raft_consensus.cc:2258] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:41.355067 9239 ts_tablet_manager.cc:1905] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 18:02:41.356383 9238 ts_tablet_manager.cc:1905] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 18:02:41.357899 9235 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260926 18:02:41.358551 9235 tablet_bootstrap.cc:492] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Bootstrap complete.
I20260926 18:02:41.359014 9235 ts_tablet_manager.cc:1397] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260926 18:02:41.361099 9235 raft_consensus.cc:348] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:41.361778 9235 raft_consensus.cc:729] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c99ae6bb45484e80360b1bb738a830, State: Initialized, Role: FOLLOWER
I20260926 18:02:41.362457 9235 consensus_queue.cc:260] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c50e5c943d40978ddd2644ca77602d" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 42083 } } peers { permanent_uuid: "a0c99ae6bb45484e80360b1bb738a830" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 36913 } } peers { permanent_uuid: "0d61f688567c4bafab9be7b985ce6f4c" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35029 } }
I20260926 18:02:41.364187 9235 ts_tablet_manager.cc:1428] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260926 18:02:41.364430 9235 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260926 18:02:41.368551 9239 ts_tablet_manager.cc:1918] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 18:02:41.368700 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 18:02:41 UTC) from {username='slave'} at 127.0.0.1:42802
I20260926 18:02:41.368953 9239 log.cc:1196] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-1-root/wals/077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.369619 9239 ts_tablet_manager.cc:1939] T 077c137bf0f945b7a9b3b0876c3009a5 P 0d61f688567c4bafab9be7b985ce6f4c: Deleting consensus metadata
I20260926 18:02:41.370003 9238 ts_tablet_manager.cc:1918] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260926 18:02:41.369968 8916 catalog_manager.cc:4821] TS a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913): delete failed for tablet 077c137bf0f945b7a9b3b0876c3009a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 077c137bf0f945b7a9b3b0876c3009a5 already in progress: copying tablet
I20260926 18:02:41.370471 9238 log.cc:1196] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-2-root/wals/077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.371186 9238 ts_tablet_manager.cc:1939] T 077c137bf0f945b7a9b3b0876c3009a5 P 61c50e5c943d40978ddd2644ca77602d: Deleting consensus metadata
I20260926 18:02:41.372179 8916 catalog_manager.cc:4841] TS 0d61f688567c4bafab9be7b985ce6f4c (127.6.90.130:35029): tablet 077c137bf0f945b7a9b3b0876c3009a5 (table test-table [id=12e000e2814b4d4cafa847143fe8579a]) successfully deleted
I20260926 18:02:41.372494 8913 catalog_manager.cc:4841] TS 61c50e5c943d40978ddd2644ca77602d (127.6.90.131:42083): tablet 077c137bf0f945b7a9b3b0876c3009a5 (table test-table [id=12e000e2814b4d4cafa847143fe8579a]) successfully deleted
I20260926 18:02:41.416185 9190 tablet_copy_service.cc:342] P 61c50e5c943d40978ddd2644ca77602d: Request end of tablet copy session a0c99ae6bb45484e80360b1bb738a830-077c137bf0f945b7a9b3b0876c3009a5 received from {username='slave'} at 127.0.0.1:58612
I20260926 18:02:41.416599 9190 tablet_copy_service.cc:433] P 61c50e5c943d40978ddd2644ca77602d: ending tablet copy session a0c99ae6bb45484e80360b1bb738a830-077c137bf0f945b7a9b3b0876c3009a5 on tablet 077c137bf0f945b7a9b3b0876c3009a5 with peer a0c99ae6bb45484e80360b1bb738a830
I20260926 18:02:41.420187 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet 077c137bf0f945b7a9b3b0876c3009a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 18:02:41 UTC) from {username='slave'} at 127.0.0.1:42802
I20260926 18:02:41.421036 9234 tablet_replica.cc:330] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: stopping tablet replica
I20260926 18:02:41.421734 9234 raft_consensus.cc:2229] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:41.422150 9234 pending_rounds.cc:62] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Trying to abort 1 pending ops.
I20260926 18:02:41.422385 9234 pending_rounds.cc:68] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333665837803347968 op_type: NO_OP noop_request { }
I20260926 18:02:41.422750 9234 raft_consensus.cc:2866] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 18:02:41.423008 9234 raft_consensus.cc:2258] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:41.424779 9234 ts_tablet_manager.cc:1905] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 18:02:41.433180 9234 ts_tablet_manager.cc:1918] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 18:02:41.433409 9234 log.cc:1196] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1790445738975945-6506-0/minicluster-data/ts-0-root/wals/077c137bf0f945b7a9b3b0876c3009a5
I20260926 18:02:41.433866 9234 ts_tablet_manager.cc:1939] T 077c137bf0f945b7a9b3b0876c3009a5 P a0c99ae6bb45484e80360b1bb738a830: Deleting consensus metadata
I20260926 18:02:41.435708 8916 catalog_manager.cc:4841] TS a0c99ae6bb45484e80360b1bb738a830 (127.6.90.129:36913): tablet 077c137bf0f945b7a9b3b0876c3009a5 (table test-table [id=12e000e2814b4d4cafa847143fe8579a]) successfully deleted
I20260926 18:02:41.485453 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:41.501753 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:41.508181 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:41.516021 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:41.529353 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:41.544539 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:41.551486 6506 tablet_server.cc:178] TabletServer@127.6.90.131:0 shutting down...
I20260926 18:02:41.565248 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:41.571000 6506 tablet_server.cc:194] TabletServer@127.6.90.131:0 shutdown complete.
I20260926 18:02:41.578441 6506 master.cc:537] Master@127.6.90.190:39059 shutting down...
I20260926 18:02:41.590968 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:41.591431 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:41.591717 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P a34856139cf14318ab378e7de2e66ef2: stopping tablet replica
I20260926 18:02:41.608932 6506 master.cc:552] Master@127.6.90.190:39059 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1268 ms)
[----------] 4 tests from TsTabletManagerITest (12235 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260926 18:02:41.631532 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:41917
I20260926 18:02:41.632539 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:41.636873 9245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:41.637887 9246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:41.639295 9248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:41.640039 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:41.640730 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:41.640887 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:41.640995 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445761640980 us; error 0 us; skew 500 ppm
I20260926 18:02:41.641412 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:41.643353 6506 webserver.cc:460] Webserver started at http://127.6.90.190:43543/ using document root <none> and password file <none>
I20260926 18:02:41.643749 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:41.643884 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:41.644068 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:41.645038 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "a6f073f4f55944c388555b976ee9bcd7"
format_stamp: "Formatted at 2026-09-26 18:02:41 on dist-test-slave-5fqf"
I20260926 18:02:41.648904 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 18:02:41.651726 9253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:41.652421 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:41.652664 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "a6f073f4f55944c388555b976ee9bcd7"
format_stamp: "Formatted at 2026-09-26 18:02:41 on dist-test-slave-5fqf"
I20260926 18:02:41.653018 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:41.663612 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:41.664511 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:41.695158 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:41917
I20260926 18:02:41.695245 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:41917 every 8 connection(s)
I20260926 18:02:41.698654 9305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:41.707376 9305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7: Bootstrap starting.
I20260926 18:02:41.710979 9305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:41.714299 9305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7: No bootstrap required, opened a new log
I20260926 18:02:41.715920 9305 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER }
I20260926 18:02:41.716266 9305 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:41.716496 9305 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f073f4f55944c388555b976ee9bcd7, State: Initialized, Role: FOLLOWER
I20260926 18:02:41.716976 9305 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER }
I20260926 18:02:41.717365 9305 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:41.717555 9305 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:41.717790 9305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:41.721398 9305 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER }
I20260926 18:02:41.721834 9305 leader_election.cc:302] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f073f4f55944c388555b976ee9bcd7; no voters:
I20260926 18:02:41.722791 9305 leader_election.cc:288] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:41.723042 9308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:41.724133 9308 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 1 LEADER]: Becoming Leader. State: Replica: a6f073f4f55944c388555b976ee9bcd7, State: Running, Role: LEADER
I20260926 18:02:41.724727 9308 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER }
I20260926 18:02:41.725271 9305 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:41.728755 9310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f073f4f55944c388555b976ee9bcd7. Latest consensus state: current_term: 1 leader_uuid: "a6f073f4f55944c388555b976ee9bcd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER } }
I20260926 18:02:41.729386 9310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:41.729405 9309 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f073f4f55944c388555b976ee9bcd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f073f4f55944c388555b976ee9bcd7" member_type: VOTER } }
I20260926 18:02:41.730897 9309 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:41.731427 9315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:41.736156 9315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:41.740407 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:41.743464 9315 catalog_manager.cc:1347] Generated new cluster ID: caf5c526c14449b8b005a778714a4fc6
I20260926 18:02:41.743714 9315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:41.760183 9315 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:41.761731 9315 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:41.775499 9315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7: Generated new TSK 0
I20260926 18:02:41.776063 9315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:41.806775 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:41.811834 9326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:41.812937 9327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:41.814090 9329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:41.814316 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:41.815079 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:41.815279 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:41.815435 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445761815423 us; error 0 us; skew 500 ppm
I20260926 18:02:41.815909 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:41.829831 6506 webserver.cc:460] Webserver started at http://127.6.90.129:35507/ using document root <none> and password file <none>
I20260926 18:02:41.830263 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:41.830411 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:41.830644 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:41.831599 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "0a383036a09a499b8bc29686fdf880cc"
format_stamp: "Formatted at 2026-09-26 18:02:41 on dist-test-slave-5fqf"
I20260926 18:02:41.835682 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260926 18:02:41.838521 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:41.839186 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:41.839425 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "0a383036a09a499b8bc29686fdf880cc"
format_stamp: "Formatted at 2026-09-26 18:02:41 on dist-test-slave-5fqf"
I20260926 18:02:41.839659 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:41.852389 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:41.853263 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:41.857774 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:41.858044 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:41.858335 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:41.858532 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:41.904878 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:37359
I20260926 18:02:41.904970 9404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:37359 every 8 connection(s)
I20260926 18:02:41.908847 9405 heartbeater.cc:347] Connected to a master server at 127.6.90.190:41917
I20260926 18:02:41.909200 9405 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:41.909787 9405 heartbeater.cc:511] Master 127.6.90.190:41917 requested a full tablet report, sending...
I20260926 18:02:41.911454 9270 ts_manager.cc:194] Registered new tserver with Master: 0a383036a09a499b8bc29686fdf880cc (127.6.90.129:37359)
I20260926 18:02:41.912482 6506 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004837295s
I20260926 18:02:41.913704 9270 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49868
I20260926 18:02:41.944415 9370 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260926 18:02:41.945534 9370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:41.955746 9415 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Bootstrap starting.
I20260926 18:02:41.959697 9415 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:41.963124 9415 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: No bootstrap required, opened a new log
I20260926 18:02:41.963459 9415 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 18:02:41.965233 9415 raft_consensus.cc:348] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:41.965619 9415 raft_consensus.cc:374] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:41.965850 9415 raft_consensus.cc:729] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a383036a09a499b8bc29686fdf880cc, State: Initialized, Role: FOLLOWER
I20260926 18:02:41.966322 9415 consensus_queue.cc:260] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:41.966754 9415 raft_consensus.cc:388] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:41.966967 9415 raft_consensus.cc:482] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:41.967221 9415 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:41.971292 9415 raft_consensus.cc:504] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:41.971845 9415 leader_election.cc:302] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a383036a09a499b8bc29686fdf880cc; no voters:
I20260926 18:02:41.973003 9415 leader_election.cc:288] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:41.973304 9417 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:41.974457 9417 raft_consensus.cc:686] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 LEADER]: Becoming Leader. State: Replica: 0a383036a09a499b8bc29686fdf880cc, State: Running, Role: LEADER
I20260926 18:02:41.974777 9405 heartbeater.cc:503] Master 127.6.90.190:41917 was elected leader, sending a full tablet report...
I20260926 18:02:41.975288 9415 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260926 18:02:41.975113 9417 consensus_queue.cc:237] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:41.979601 9418 tablet_replica.cc:439] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a383036a09a499b8bc29686fdf880cc" committed_config { opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } } }
I20260926 18:02:41.979775 9419 tablet_replica.cc:439] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: TxnStatusTablet state changed. Reason: New leader 0a383036a09a499b8bc29686fdf880cc. Latest consensus state: current_term: 1 leader_uuid: "0a383036a09a499b8bc29686fdf880cc" committed_config { opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } } }
I20260926 18:02:41.980314 9419 tablet_replica.cc:442] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:41.981670 9418 tablet_replica.cc:442] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:41.981778 9422 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:41.982250 9422 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260926 18:02:41.982297 9270 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260926 18:02:41.985966 9270 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:42.017724 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:42.041720 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:42.042476 6506 tablet_replica.cc:330] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: stopping tablet replica
I20260926 18:02:42.042950 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:42.043349 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:42.063431 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:42.075832 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:42.081984 9425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.082176 9426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.084642 9428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.085187 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:42.086223 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:42.086464 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:42.086651 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445762086636 us; error 0 us; skew 500 ppm
I20260926 18:02:42.087291 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:42.090054 6506 webserver.cc:460] Webserver started at http://127.6.90.129:35507/ using document root <none> and password file <none>
I20260926 18:02:42.090631 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:42.090835 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:42.095301 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 18:02:42.098441 9433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.099227 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:42.099493 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "0a383036a09a499b8bc29686fdf880cc"
format_stamp: "Formatted at 2026-09-26 18:02:41 on dist-test-slave-5fqf"
I20260926 18:02:42.099749 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:42.116663 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:42.117579 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:42.124083 9442 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 18:02:42.131379 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 18:02:42.131677 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.002s sys 0.000s
I20260926 18:02:42.131989 6506 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 18:02:42.137025 6506 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 18:02:42.137283 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.006s sys 0.000s
I20260926 18:02:42.137351 9442 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Bootstrap starting.
I20260926 18:02:42.173517 9442 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:42.174621 9442 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Bootstrap complete.
I20260926 18:02:42.175374 9442 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Time spent bootstrapping tablet: real 0.038s user 0.023s sys 0.014s
I20260926 18:02:42.178284 9442 raft_consensus.cc:348] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:42.179018 9442 raft_consensus.cc:729] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a383036a09a499b8bc29686fdf880cc, State: Initialized, Role: FOLLOWER
I20260926 18:02:42.179819 9442 consensus_queue.cc:260] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:42.180649 9442 raft_consensus.cc:388] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:42.181137 9442 raft_consensus.cc:482] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:42.181535 9442 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:42.188557 9442 raft_consensus.cc:504] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:42.189360 9442 leader_election.cc:302] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a383036a09a499b8bc29686fdf880cc; no voters:
I20260926 18:02:42.197288 9455 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:42.197710 9442 leader_election.cc:288] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 18:02:42.201423 9455 raft_consensus.cc:686] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 2 LEADER]: Becoming Leader. State: Replica: 0a383036a09a499b8bc29686fdf880cc, State: Running, Role: LEADER
I20260926 18:02:42.202322 9455 consensus_queue.cc:237] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } }
I20260926 18:02:42.207875 9455 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0a383036a09a499b8bc29686fdf880cc" committed_config { opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } } }
I20260926 18:02:42.209784 9455 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:42.209185 9460 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 0a383036a09a499b8bc29686fdf880cc. Latest consensus state: current_term: 2 leader_uuid: "0a383036a09a499b8bc29686fdf880cc" committed_config { opid_index: -1 peers { permanent_uuid: "0a383036a09a499b8bc29686fdf880cc" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37359 } } }
I20260926 18:02:42.210844 9460 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:42.216511 9442 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc: Time spent starting tablet: real 0.041s user 0.007s sys 0.031s
I20260926 18:02:42.217489 9463 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:42.218015 9463 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 18:02:42.263718 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:37359
I20260926 18:02:42.263834 9511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:37359 every 8 connection(s)
I20260926 18:02:42.269024 9512 heartbeater.cc:347] Connected to a master server at 127.6.90.190:41917
I20260926 18:02:42.269451 9512 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:42.270244 9512 heartbeater.cc:511] Master 127.6.90.190:41917 requested a full tablet report, sending...
I20260926 18:02:42.272624 6506 tablet_server.cc:178] TabletServer@127.6.90.129:37359 shutting down...
I20260926 18:02:42.273515 9270 ts_manager.cc:194] Re-registered known tserver with Master: 0a383036a09a499b8bc29686fdf880cc (127.6.90.129:37359)
W20260926 18:02:42.274201 9270 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:42.275899 9270 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49890
I20260926 18:02:42.279524 9512 heartbeater.cc:503] Master 127.6.90.190:41917 was elected leader, sending a full tablet report...
W20260926 18:02:42.282375 9270 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:42.299328 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:42.300427 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:42.301069 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:42.301569 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 0a383036a09a499b8bc29686fdf880cc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:42.312206 6506 tablet_server.cc:194] TabletServer@127.6.90.129:37359 shutdown complete.
I20260926 18:02:42.321878 6506 master.cc:537] Master@127.6.90.190:41917 shutting down...
I20260926 18:02:42.338027 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:42.338470 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:42.338759 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P a6f073f4f55944c388555b976ee9bcd7: stopping tablet replica
I20260926 18:02:42.355792 6506 master.cc:552] Master@127.6.90.190:41917 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (744 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260926 18:02:42.377619 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:33201
I20260926 18:02:42.378743 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:42.384261 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.384611 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.387036 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:42.387560 9521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.388317 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:42.388538 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:42.388681 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445762388667 us; error 0 us; skew 500 ppm
I20260926 18:02:42.389158 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:42.391428 6506 webserver.cc:460] Webserver started at http://127.6.90.190:35407/ using document root <none> and password file <none>
I20260926 18:02:42.391860 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:42.392020 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:42.392246 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:42.393244 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "7a91a6b48ff34c6194a16a4b642ff70a"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.397418 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 18:02:42.400597 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.401321 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:42.401567 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "7a91a6b48ff34c6194a16a4b642ff70a"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.401831 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:42.423988 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:42.424932 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:42.463491 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:33201
I20260926 18:02:42.463626 9577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:33201 every 8 connection(s)
I20260926 18:02:42.467440 9578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:42.480822 9578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a: Bootstrap starting.
I20260926 18:02:42.486099 9578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:42.490429 9578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a: No bootstrap required, opened a new log
I20260926 18:02:42.492739 9578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER }
I20260926 18:02:42.493209 9578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:42.493481 9578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a91a6b48ff34c6194a16a4b642ff70a, State: Initialized, Role: FOLLOWER
I20260926 18:02:42.494093 9578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER }
I20260926 18:02:42.494655 9578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:42.494935 9578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:42.495229 9578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:42.501047 9578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER }
I20260926 18:02:42.501647 9578 leader_election.cc:302] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a91a6b48ff34c6194a16a4b642ff70a; no voters:
I20260926 18:02:42.503181 9581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:42.502885 9578 leader_election.cc:288] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:42.504561 9581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 1 LEADER]: Becoming Leader. State: Replica: 7a91a6b48ff34c6194a16a4b642ff70a, State: Running, Role: LEADER
I20260926 18:02:42.505335 9581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER }
I20260926 18:02:42.507138 9578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:42.507992 9582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a91a6b48ff34c6194a16a4b642ff70a. Latest consensus state: current_term: 1 leader_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER } }
I20260926 18:02:42.508154 9583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a91a6b48ff34c6194a16a4b642ff70a" member_type: VOTER } }
I20260926 18:02:42.508533 9582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:42.508615 9583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:42.510171 9585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:42.516870 9585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:42.528863 9585 catalog_manager.cc:1347] Generated new cluster ID: 03a53c2f49a34c1b8b409459fda35c01
I20260926 18:02:42.529115 9585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:42.534183 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:42.543361 9585 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:42.544891 9585 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:42.559621 9585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a: Generated new TSK 0
I20260926 18:02:42.560292 9585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:42.568260 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:42.574584 9599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.575227 9600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.577880 9602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.578074 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:42.579056 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:42.579278 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:42.579435 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445762579425 us; error 0 us; skew 500 ppm
I20260926 18:02:42.580016 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:42.582532 6506 webserver.cc:460] Webserver started at http://127.6.90.129:45841/ using document root <none> and password file <none>
I20260926 18:02:42.583112 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:42.583324 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:42.583636 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:42.585031 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "fcb444a4ce35417e88ae4c7329264fad"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.590548 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 18:02:42.594372 9607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.595147 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:42.595455 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "fcb444a4ce35417e88ae4c7329264fad"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.595786 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:42.619625 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:42.620819 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:42.638980 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:42.639312 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.639597 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:42.639832 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.701655 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:45411
I20260926 18:02:42.701715 9677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:45411 every 8 connection(s)
I20260926 18:02:42.706921 9678 heartbeater.cc:347] Connected to a master server at 127.6.90.190:33201
I20260926 18:02:42.707276 9678 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:42.707429 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 18:02:42.708033 9678 heartbeater.cc:511] Master 127.6.90.190:33201 requested a full tablet report, sending...
I20260926 18:02:42.709832 9543 ts_manager.cc:194] Registered new tserver with Master: fcb444a4ce35417e88ae4c7329264fad (127.6.90.129:45411)
I20260926 18:02:42.711524 9543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41458
W20260926 18:02:42.714349 9680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:42.716542 9681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.722510 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:42.722711 9683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:42.723673 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:42.723922 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:42.724108 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445762724094 us; error 0 us; skew 500 ppm
I20260926 18:02:42.724774 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:42.727306 6506 webserver.cc:460] Webserver started at http://127.6.90.130:35407/ using document root <none> and password file <none>
I20260926 18:02:42.727880 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:42.728092 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:42.728410 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:42.729705 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "283b0937c71b411ca6321c0c1eda6129"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.735277 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 18:02:42.739022 9688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.739771 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 18:02:42.740077 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "283b0937c71b411ca6321c0c1eda6129"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:42.740432 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:42.768538 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:42.769802 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:42.775646 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:42.775908 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:42.776170 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:42.776377 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 18:02:42.849201 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:44207
I20260926 18:02:42.849288 9757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:44207 every 8 connection(s)
I20260926 18:02:42.854540 9758 heartbeater.cc:347] Connected to a master server at 127.6.90.190:33201
I20260926 18:02:42.854934 9758 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:42.855767 9758 heartbeater.cc:511] Master 127.6.90.190:33201 requested a full tablet report, sending...
I20260926 18:02:42.857746 9543 ts_manager.cc:194] Registered new tserver with Master: 283b0937c71b411ca6321c0c1eda6129 (127.6.90.130:44207)
I20260926 18:02:42.858436 6506 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.004884717s
I20260926 18:02:42.859930 9543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41486
I20260926 18:02:42.894376 9643 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260926 18:02:42.895524 9643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:42.905886 9768 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Bootstrap starting.
I20260926 18:02:42.911363 9768 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:42.915997 9768 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: No bootstrap required, opened a new log
I20260926 18:02:42.916476 9768 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260926 18:02:42.918789 9768 raft_consensus.cc:348] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:42.919312 9768 raft_consensus.cc:374] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:42.919582 9768 raft_consensus.cc:729] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb444a4ce35417e88ae4c7329264fad, State: Initialized, Role: FOLLOWER
I20260926 18:02:42.920159 9768 consensus_queue.cc:260] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:42.920779 9768 raft_consensus.cc:388] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:42.921058 9768 raft_consensus.cc:482] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:42.921363 9768 raft_consensus.cc:3037] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:42.957144 9768 raft_consensus.cc:504] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:42.957894 9768 leader_election.cc:302] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb444a4ce35417e88ae4c7329264fad; no voters:
I20260926 18:02:42.959360 9768 leader_election.cc:288] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:42.959817 9770 raft_consensus.cc:2781] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:42.961323 9678 heartbeater.cc:503] Master 127.6.90.190:33201 was elected leader, sending a full tablet report...
I20260926 18:02:42.962041 9768 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Time spent starting tablet: real 0.045s user 0.013s sys 0.002s
I20260926 18:02:42.962080 9770 raft_consensus.cc:686] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 LEADER]: Becoming Leader. State: Replica: fcb444a4ce35417e88ae4c7329264fad, State: Running, Role: LEADER
I20260926 18:02:42.962852 9770 consensus_queue.cc:237] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:42.965991 9771 tablet_replica.cc:439] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb444a4ce35417e88ae4c7329264fad" committed_config { opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } } }
I20260926 18:02:42.966168 9772 tablet_replica.cc:439] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: TxnStatusTablet state changed. Reason: New leader fcb444a4ce35417e88ae4c7329264fad. Latest consensus state: current_term: 1 leader_uuid: "fcb444a4ce35417e88ae4c7329264fad" committed_config { opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } } }
I20260926 18:02:42.966513 9771 tablet_replica.cc:442] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:42.966717 9772 tablet_replica.cc:442] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: This TxnStatusTablet replica's current role is: LEADER
W20260926 18:02:42.970840 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:42.976130 9774 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:42.976558 9774 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260926 18:02:42.977722 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:43.028133 9779 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Initiating tablet copy from peer fcb444a4ce35417e88ae4c7329264fad (127.6.90.129:45411)
I20260926 18:02:43.029757 9779 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: tablet copy: Beginning tablet copy session from remote peer at address 127.6.90.129:45411
I20260926 18:02:43.038872 9663 tablet_copy_service.cc:140] P fcb444a4ce35417e88ae4c7329264fad: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer 283b0937c71b411ca6321c0c1eda6129 ({username='slave'} at 127.0.0.1:58902)
I20260926 18:02:43.039273 9663 tablet_copy_service.cc:161] P fcb444a4ce35417e88ae4c7329264fad: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer 283b0937c71b411ca6321c0c1eda6129 at {username='slave'} at 127.0.0.1:58902: session id = 283b0937c71b411ca6321c0c1eda6129-11111111111111111111111111111111
I20260926 18:02:43.043934 9663 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 18:02:43.046219 9779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:43.055284 9779 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: tablet copy: Starting download of 0 data blocks...
I20260926 18:02:43.055752 9779 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: tablet copy: Starting download of 1 WAL segments...
I20260926 18:02:43.058910 9779 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 18:02:43.148289 9779 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Bootstrap starting.
I20260926 18:02:43.168257 9779 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:43.169013 9779 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Bootstrap complete.
I20260926 18:02:43.169481 9779 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.004s
I20260926 18:02:43.171273 9779 raft_consensus.cc:348] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.171689 9779 raft_consensus.cc:729] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 283b0937c71b411ca6321c0c1eda6129, State: Initialized, Role: NON_PARTICIPANT
I20260926 18:02:43.172163 9779 consensus_queue.cc:260] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.173813 9782 tablet_replica.cc:439] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } } }
I20260926 18:02:43.174037 9758 heartbeater.cc:503] Master 127.6.90.190:33201 was elected leader, sending a full tablet report...
I20260926 18:02:43.174459 9779 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 18:02:43.174463 9782 tablet_replica.cc:442] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260926 18:02:43.176185 9663 tablet_copy_service.cc:342] P fcb444a4ce35417e88ae4c7329264fad: Request end of tablet copy session 283b0937c71b411ca6321c0c1eda6129-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:58902
I20260926 18:02:43.176528 9663 tablet_copy_service.cc:433] P fcb444a4ce35417e88ae4c7329264fad: ending tablet copy session 283b0937c71b411ca6321c0c1eda6129-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer 283b0937c71b411ca6321c0c1eda6129
W20260926 18:02:43.177098 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260926 18:02:43.180876 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:43.184067 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:43.202780 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:43.204171 6506 tablet_replica.cc:330] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: stopping tablet replica
I20260926 18:02:43.205061 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:43.205586 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:43.215634 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:43.225826 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:43.231309 9783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:43.232486 9784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:43.235303 9786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:43.236068 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:43.236873 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:43.237052 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:43.237212 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445763237199 us; error 0 us; skew 500 ppm
I20260926 18:02:43.237684 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:43.240453 6506 webserver.cc:460] Webserver started at http://127.6.90.129:45841/ using document root <none> and password file <none>
I20260926 18:02:43.240927 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:43.241091 6506 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 18:02:43.244673 6506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 18:02:43.247328 9791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:43.248133 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:43.248492 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "fcb444a4ce35417e88ae4c7329264fad"
format_stamp: "Formatted at 2026-09-26 18:02:42 on dist-test-slave-5fqf"
I20260926 18:02:43.248847 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:43.266440 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:43.267655 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:43.275067 9800 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 18:02:43.281876 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 18:02:43.282183 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.001s
I20260926 18:02:43.282481 6506 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 18:02:43.289177 6506 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 18:02:43.289463 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.008s sys 0.000s
I20260926 18:02:43.289491 9800 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Bootstrap starting.
I20260926 18:02:43.324381 9800 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:43.325279 9800 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Bootstrap complete.
I20260926 18:02:43.326027 9800 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Time spent bootstrapping tablet: real 0.037s user 0.018s sys 0.014s
I20260926 18:02:43.328737 9800 raft_consensus.cc:348] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.329283 9800 raft_consensus.cc:729] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb444a4ce35417e88ae4c7329264fad, State: Initialized, Role: FOLLOWER
I20260926 18:02:43.330011 9800 consensus_queue.cc:260] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.330672 9800 raft_consensus.cc:388] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:43.330953 9800 raft_consensus.cc:482] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:43.331310 9800 raft_consensus.cc:3037] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:43.356056 9800 raft_consensus.cc:504] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.356799 9800 leader_election.cc:302] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb444a4ce35417e88ae4c7329264fad; no voters:
I20260926 18:02:43.360800 9800 leader_election.cc:288] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 18:02:43.361079 9849 raft_consensus.cc:2781] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:43.371209 9849 raft_consensus.cc:686] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 2 LEADER]: Becoming Leader. State: Replica: fcb444a4ce35417e88ae4c7329264fad, State: Running, Role: LEADER
I20260926 18:02:43.371464 9800 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad: Time spent starting tablet: real 0.045s user 0.018s sys 0.006s
I20260926 18:02:43.372017 9849 consensus_queue.cc:237] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } }
I20260926 18:02:43.383199 9851 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fcb444a4ce35417e88ae4c7329264fad" committed_config { opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } } }
I20260926 18:02:43.384451 9851 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:43.383888 9852 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader fcb444a4ce35417e88ae4c7329264fad. Latest consensus state: current_term: 2 leader_uuid: "fcb444a4ce35417e88ae4c7329264fad" committed_config { opid_index: -1 peers { permanent_uuid: "fcb444a4ce35417e88ae4c7329264fad" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45411 } } }
I20260926 18:02:43.386123 9852 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:43.400277 9861 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:43.400825 9861 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 18:02:43.405908 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:45411
I20260926 18:02:43.406023 9869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:45411 every 8 connection(s)
I20260926 18:02:43.411563 9870 heartbeater.cc:347] Connected to a master server at 127.6.90.190:33201
I20260926 18:02:43.412037 9870 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:43.412940 9870 heartbeater.cc:511] Master 127.6.90.190:33201 requested a full tablet report, sending...
I20260926 18:02:43.415187 6506 tablet_server.cc:178] TabletServer@127.6.90.129:45411 shutting down...
I20260926 18:02:43.416096 9543 ts_manager.cc:194] Re-registered known tserver with Master: fcb444a4ce35417e88ae4c7329264fad (127.6.90.129:45411)
W20260926 18:02:43.417361 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:43.420419 9543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41534
I20260926 18:02:43.425776 9870 heartbeater.cc:503] Master 127.6.90.190:33201 was elected leader, sending a full tablet report...
W20260926 18:02:43.428956 9543 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:43.439105 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:43.439806 6506 tablet_replica.cc:330] stopping tablet replica
I20260926 18:02:43.440300 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:43.440794 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P fcb444a4ce35417e88ae4c7329264fad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:43.451824 6506 tablet_server.cc:194] TabletServer@127.6.90.129:45411 shutdown complete.
I20260926 18:02:43.463728 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:43.481308 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:43.482088 6506 tablet_replica.cc:330] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129: stopping tablet replica
I20260926 18:02:43.482689 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260926 18:02:43.483139 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 283b0937c71b411ca6321c0c1eda6129 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260926 18:02:43.503046 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:43.515806 6506 master.cc:537] Master@127.6.90.190:33201 shutting down...
I20260926 18:02:43.537050 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:43.537497 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:43.537765 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7a91a6b48ff34c6194a16a4b642ff70a: stopping tablet replica
I20260926 18:02:43.555315 6506 master.cc:552] Master@127.6.90.190:33201 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1200 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260926 18:02:43.577476 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:38893
I20260926 18:02:43.578433 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:43.583344 9876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:43.585047 9877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:43.586800 9879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:43.587428 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:43.588037 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:43.588196 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:43.588325 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445763588315 us; error 0 us; skew 500 ppm
I20260926 18:02:43.588891 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:43.591408 6506 webserver.cc:460] Webserver started at http://127.6.90.190:40649/ using document root <none> and password file <none>
I20260926 18:02:43.591832 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:43.591980 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:43.592206 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:43.593185 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "66d12d36c5074d4e9f764f9d378dbe09"
format_stamp: "Formatted at 2026-09-26 18:02:43 on dist-test-slave-5fqf"
I20260926 18:02:43.597345 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 18:02:43.600445 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:43.601214 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:43.601521 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "66d12d36c5074d4e9f764f9d378dbe09"
format_stamp: "Formatted at 2026-09-26 18:02:43 on dist-test-slave-5fqf"
I20260926 18:02:43.601862 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:43.661855 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:43.663172 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:43.708098 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:38893
I20260926 18:02:43.708169 9935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:38893 every 8 connection(s)
I20260926 18:02:43.712267 9936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:43.726792 9936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09: Bootstrap starting.
I20260926 18:02:43.732290 9936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:43.736737 9936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09: No bootstrap required, opened a new log
I20260926 18:02:43.739091 9936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER }
I20260926 18:02:43.739571 9936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:43.739845 9936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d12d36c5074d4e9f764f9d378dbe09, State: Initialized, Role: FOLLOWER
I20260926 18:02:43.740477 9936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER }
I20260926 18:02:43.741057 9936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:43.741345 9936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:43.741649 9936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:43.772176 9936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER }
I20260926 18:02:43.772781 9936 leader_election.cc:302] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66d12d36c5074d4e9f764f9d378dbe09; no voters:
I20260926 18:02:43.774057 9936 leader_election.cc:288] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:43.774509 9939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:43.776907 9939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 1 LEADER]: Becoming Leader. State: Replica: 66d12d36c5074d4e9f764f9d378dbe09, State: Running, Role: LEADER
I20260926 18:02:43.777624 9939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER }
I20260926 18:02:43.778149 9936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:43.780732 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66d12d36c5074d4e9f764f9d378dbe09. Latest consensus state: current_term: 1 leader_uuid: "66d12d36c5074d4e9f764f9d378dbe09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER } }
I20260926 18:02:43.780766 9940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66d12d36c5074d4e9f764f9d378dbe09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d12d36c5074d4e9f764f9d378dbe09" member_type: VOTER } }
I20260926 18:02:43.781494 9940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:43.781384 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:43.791785 9944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:43.796710 9944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:43.799481 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:43.806738 9944 catalog_manager.cc:1347] Generated new cluster ID: 809e95dfdc8b4ff598e4f0e5b03a874a
I20260926 18:02:43.806970 9944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:43.820111 9944 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:43.821602 9944 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:43.833211 9944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09: Generated new TSK 0
I20260926 18:02:43.833770 9944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:43.868412 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:43.874017 9957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:43.876035 9958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:43.883194 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:43.883251 9960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:43.884251 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:43.884492 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:43.884696 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445763884677 us; error 0 us; skew 500 ppm
I20260926 18:02:43.885165 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:43.888286 6506 webserver.cc:460] Webserver started at http://127.6.90.129:39869/ using document root <none> and password file <none>
I20260926 18:02:43.888770 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:43.888931 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:43.889160 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:43.890161 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "ec8a166a3501400baf97ac081a927ed1"
format_stamp: "Formatted at 2026-09-26 18:02:43 on dist-test-slave-5fqf"
I20260926 18:02:43.894562 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 18:02:43.897670 9965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:43.898379 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 18:02:43.898622 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "ec8a166a3501400baf97ac081a927ed1"
format_stamp: "Formatted at 2026-09-26 18:02:43 on dist-test-slave-5fqf"
I20260926 18:02:43.898865 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:43.916869 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:43.917922 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:43.923163 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:43.923411 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 18:02:43.923674 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:43.923861 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:43.992434 10035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:38199 every 8 connection(s)
I20260926 18:02:43.993053 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:38199
I20260926 18:02:43.998456 10036 heartbeater.cc:347] Connected to a master server at 127.6.90.190:38893
I20260926 18:02:43.998859 10036 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:43.999696 10036 heartbeater.cc:511] Master 127.6.90.190:38893 requested a full tablet report, sending...
I20260926 18:02:44.001829 9901 ts_manager.cc:194] Registered new tserver with Master: ec8a166a3501400baf97ac081a927ed1 (127.6.90.129:38199)
I20260926 18:02:44.002206 6506 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004819837s
I20260926 18:02:44.004577 9901 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46704
I20260926 18:02:44.038959 10000 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260926 18:02:44.040066 10000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:44.050221 10046 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: Bootstrap starting.
I20260926 18:02:44.055534 10046 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:44.060829 10046 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: No bootstrap required, opened a new log
I20260926 18:02:44.061272 10046 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260926 18:02:44.063544 10046 raft_consensus.cc:348] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } }
I20260926 18:02:44.064225 10046 raft_consensus.cc:374] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:44.064539 10046 raft_consensus.cc:729] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8a166a3501400baf97ac081a927ed1, State: Initialized, Role: FOLLOWER
I20260926 18:02:44.065160 10046 consensus_queue.cc:260] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } }
I20260926 18:02:44.065778 10046 raft_consensus.cc:388] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:44.066058 10046 raft_consensus.cc:482] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:44.066380 10046 raft_consensus.cc:3037] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:44.078377 10046 raft_consensus.cc:504] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } }
I20260926 18:02:44.079169 10046 leader_election.cc:302] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8a166a3501400baf97ac081a927ed1; no voters:
I20260926 18:02:44.080765 10046 leader_election.cc:288] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:44.081185 10048 raft_consensus.cc:2781] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:44.082962 10036 heartbeater.cc:503] Master 127.6.90.190:38893 was elected leader, sending a full tablet report...
I20260926 18:02:44.083501 10046 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: Time spent starting tablet: real 0.022s user 0.011s sys 0.004s
I20260926 18:02:44.083688 10048 raft_consensus.cc:686] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 1 LEADER]: Becoming Leader. State: Replica: ec8a166a3501400baf97ac081a927ed1, State: Running, Role: LEADER
I20260926 18:02:44.084450 10048 consensus_queue.cc:237] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } }
I20260926 18:02:44.087821 10049 tablet_replica.cc:439] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8a166a3501400baf97ac081a927ed1" committed_config { opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } } }
I20260926 18:02:44.088438 10049 tablet_replica.cc:442] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:44.089380 10050 tablet_replica.cc:439] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: TxnStatusTablet state changed. Reason: New leader ec8a166a3501400baf97ac081a927ed1. Latest consensus state: current_term: 1 leader_uuid: "ec8a166a3501400baf97ac081a927ed1" committed_config { opid_index: -1 peers { permanent_uuid: "ec8a166a3501400baf97ac081a927ed1" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 38199 } } }
I20260926 18:02:44.089986 10050 tablet_replica.cc:442] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: This TxnStatusTablet replica's current role is: LEADER
W20260926 18:02:44.090459 9901 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:44.092397 10052 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:44.092952 10052 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260926 18:02:44.095129 9901 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260926 18:02:44.156092 9995 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260926 18:02:44.156394 9995 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260926 18:02:44.156684 9995 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260926 18:02:44.164721 9995 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260926 18:02:44.207135 9995 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260926 18:02:44.207528 9995 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260926 18:02:44.215188 9995 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260926 18:02:44.228240 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.230299 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.232036 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.233748 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.235376 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.237134 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:42580
W20260926 18:02:44.248111 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
W20260926 18:02:44.249934 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
W20260926 18:02:44.251616 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
W20260926 18:02:44.253242 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
W20260926 18:02:44.254739 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
W20260926 18:02:44.256194 9999 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:42588
I20260926 18:02:44.258335 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:44.275085 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:44.275772 6506 tablet_replica.cc:330] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1: stopping tablet replica
I20260926 18:02:44.276285 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:44.276788 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P ec8a166a3501400baf97ac081a927ed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:44.296262 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:44.305944 6506 master.cc:537] Master@127.6.90.190:38893 shutting down...
I20260926 18:02:44.318331 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:44.318776 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:44.319097 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 66d12d36c5074d4e9f764f9d378dbe09: stopping tablet replica
I20260926 18:02:44.335387 6506 master.cc:552] Master@127.6.90.190:38893 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (782 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260926 18:02:44.358791 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:46599
I20260926 18:02:44.359710 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:44.363994 10061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:44.365020 10062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:44.367869 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:44.368203 10064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:44.368917 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:44.369097 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:44.369210 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445764369202 us; error 0 us; skew 500 ppm
I20260926 18:02:44.369627 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:44.371567 6506 webserver.cc:460] Webserver started at http://127.6.90.190:36939/ using document root <none> and password file <none>
I20260926 18:02:44.371977 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:44.372115 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:44.372304 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:44.373266 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "85116f54c897489395a22b744f423385"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.377094 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 18:02:44.379865 10069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:44.380504 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:44.380755 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "85116f54c897489395a22b744f423385"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.380985 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:44.407891 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:44.408867 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:44.440035 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:46599
I20260926 18:02:44.440117 10120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:46599 every 8 connection(s)
I20260926 18:02:44.443397 10121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:44.452101 10121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385: Bootstrap starting.
I20260926 18:02:44.455864 10121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:44.459285 10121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385: No bootstrap required, opened a new log
I20260926 18:02:44.460968 10121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER }
I20260926 18:02:44.461321 10121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:44.461488 10121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85116f54c897489395a22b744f423385, State: Initialized, Role: FOLLOWER
I20260926 18:02:44.461975 10121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER }
I20260926 18:02:44.462363 10121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:44.462529 10121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:44.462729 10121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:44.466364 10121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER }
I20260926 18:02:44.466774 10121 leader_election.cc:302] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85116f54c897489395a22b744f423385; no voters:
I20260926 18:02:44.467711 10121 leader_election.cc:288] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:44.467967 10124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:44.469103 10124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 1 LEADER]: Becoming Leader. State: Replica: 85116f54c897489395a22b744f423385, State: Running, Role: LEADER
I20260926 18:02:44.469702 10124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER }
I20260926 18:02:44.470458 10121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:44.473124 10125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85116f54c897489395a22b744f423385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER } }
I20260926 18:02:44.473688 10125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:44.473206 10126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85116f54c897489395a22b744f423385. Latest consensus state: current_term: 1 leader_uuid: "85116f54c897489395a22b744f423385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85116f54c897489395a22b744f423385" member_type: VOTER } }
I20260926 18:02:44.474393 10126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:44.476392 10131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:44.483160 10131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:44.484076 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:44.490594 10131 catalog_manager.cc:1347] Generated new cluster ID: 721860320c1d4c6c9c9290f3176357c1
I20260926 18:02:44.490793 10131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:44.509752 10131 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:44.511194 10131 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:44.522624 10131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385: Generated new TSK 0
I20260926 18:02:44.523152 10131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:44.550082 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:44.555437 10142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:44.556361 10143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:44.558387 10145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:44.558869 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:44.559610 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:44.559774 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:44.559892 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445764559883 us; error 0 us; skew 500 ppm
I20260926 18:02:44.560329 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:44.562433 6506 webserver.cc:460] Webserver started at http://127.6.90.129:43475/ using document root <none> and password file <none>
I20260926 18:02:44.562836 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:44.562973 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:44.563174 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:44.564131 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "be61bf07094a43e9ae3fa0b325ff4ca9"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.568089 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:44.571026 10150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:44.571714 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:44.571939 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "be61bf07094a43e9ae3fa0b325ff4ca9"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.572192 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:44.598161 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:44.599054 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:44.603298 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:44.603564 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:44.603857 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:44.604064 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:44.648882 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:37473
I20260926 18:02:44.648960 10220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:37473 every 8 connection(s)
I20260926 18:02:44.652789 10221 heartbeater.cc:347] Connected to a master server at 127.6.90.190:46599
I20260926 18:02:44.653105 10221 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:44.653678 10221 heartbeater.cc:511] Master 127.6.90.190:46599 requested a full tablet report, sending...
I20260926 18:02:44.655283 10086 ts_manager.cc:194] Registered new tserver with Master: be61bf07094a43e9ae3fa0b325ff4ca9 (127.6.90.129:37473)
I20260926 18:02:44.655375 6506 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003754093s
I20260926 18:02:44.657454 10086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38158
I20260926 18:02:44.685722 10186 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260926 18:02:44.686721 10186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:44.696274 10231 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: Bootstrap starting.
I20260926 18:02:44.699873 10231 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:44.703583 10231 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: No bootstrap required, opened a new log
I20260926 18:02:44.703890 10231 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260926 18:02:44.705555 10231 raft_consensus.cc:348] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.705922 10231 raft_consensus.cc:374] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:44.706130 10231 raft_consensus.cc:729] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be61bf07094a43e9ae3fa0b325ff4ca9, State: Initialized, Role: FOLLOWER
I20260926 18:02:44.706593 10231 consensus_queue.cc:260] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.707013 10231 raft_consensus.cc:388] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:44.707214 10231 raft_consensus.cc:482] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:44.707449 10231 raft_consensus.cc:3037] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:44.711195 10231 raft_consensus.cc:504] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.711648 10231 leader_election.cc:302] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be61bf07094a43e9ae3fa0b325ff4ca9; no voters:
I20260926 18:02:44.712632 10231 leader_election.cc:288] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:44.712901 10233 raft_consensus.cc:2781] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:44.714044 10233 raft_consensus.cc:686] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 LEADER]: Becoming Leader. State: Replica: be61bf07094a43e9ae3fa0b325ff4ca9, State: Running, Role: LEADER
I20260926 18:02:44.714287 10221 heartbeater.cc:503] Master 127.6.90.190:46599 was elected leader, sending a full tablet report...
I20260926 18:02:44.714636 10231 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260926 18:02:44.714713 10233 consensus_queue.cc:237] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.720062 10235 tablet_replica.cc:439] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: TxnStatusTablet state changed. Reason: New leader be61bf07094a43e9ae3fa0b325ff4ca9. Latest consensus state: current_term: 1 leader_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" committed_config { opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } } }
I20260926 18:02:44.720155 10234 tablet_replica.cc:439] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" committed_config { opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } } }
I20260926 18:02:44.720608 10235 tablet_replica.cc:442] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: This TxnStatusTablet replica's current role is: LEADER
I20260926 18:02:44.720686 10234 tablet_replica.cc:442] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: This TxnStatusTablet replica's current role is: LEADER
W20260926 18:02:44.721473 10086 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:44.721922 10238 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 18:02:44.722258 10238 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260926 18:02:44.724881 10086 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260926 18:02:44.742053 10186 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260926 18:02:44.742985 10186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:44.751557 10231 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: Bootstrap starting.
I20260926 18:02:44.755256 10231 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:44.758854 10231 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: No bootstrap required, opened a new log
I20260926 18:02:44.759182 10231 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 18:02:44.760828 10231 raft_consensus.cc:348] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.761195 10231 raft_consensus.cc:374] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:44.761391 10231 raft_consensus.cc:729] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be61bf07094a43e9ae3fa0b325ff4ca9, State: Initialized, Role: FOLLOWER
I20260926 18:02:44.761817 10231 consensus_queue.cc:260] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.762224 10231 raft_consensus.cc:388] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:44.762416 10231 raft_consensus.cc:482] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:44.762663 10231 raft_consensus.cc:3037] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:44.766433 10231 raft_consensus.cc:504] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.766881 10231 leader_election.cc:302] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be61bf07094a43e9ae3fa0b325ff4ca9; no voters:
I20260926 18:02:44.767283 10231 leader_election.cc:288] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:44.767443 10235 raft_consensus.cc:2781] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:44.767889 10235 raft_consensus.cc:686] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 LEADER]: Becoming Leader. State: Replica: be61bf07094a43e9ae3fa0b325ff4ca9, State: Running, Role: LEADER
I20260926 18:02:44.768489 10235 consensus_queue.cc:237] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "be61bf07094a43e9ae3fa0b325ff4ca9" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 37473 } }
I20260926 18:02:44.768882 10231 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
W20260926 18:02:44.773151 10086 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260926 18:02:44.776396 10086 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260926 18:02:44.798056 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:44.820379 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:44.821035 6506 tablet_replica.cc:330] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9: stopping tablet replica
I20260926 18:02:44.821472 6506 raft_consensus.cc:2229] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:44.821825 6506 raft_consensus.cc:2258] T 22222222222222222222222222222222 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:44.823982 6506 tablet_replica.cc:330] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9: stopping tablet replica
I20260926 18:02:44.824463 6506 raft_consensus.cc:2229] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:44.824815 6506 raft_consensus.cc:2258] T 11111111111111111111111111111111 P be61bf07094a43e9ae3fa0b325ff4ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:44.834336 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:44.843886 6506 master.cc:537] Master@127.6.90.190:46599 shutting down...
I20260926 18:02:44.856118 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:44.856616 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:44.856984 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 85116f54c897489395a22b744f423385: stopping tablet replica
I20260926 18:02:44.874321 6506 master.cc:552] Master@127.6.90.190:46599 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (536 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3263 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260926 18:02:44.895380 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:36873
I20260926 18:02:44.896483 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:44.900993 10247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:44.901897 10248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:44.903261 10250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:44.903338 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:44.904210 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:44.904402 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:44.904525 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445764904516 us; error 0 us; skew 500 ppm
I20260926 18:02:44.904948 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:44.906968 6506 webserver.cc:460] Webserver started at http://127.6.90.190:45195/ using document root <none> and password file <none>
I20260926 18:02:44.907413 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:44.907554 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:44.907752 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:44.908918 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "5f5e173144b047998f485728e7395258"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.912940 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 18:02:44.915889 10255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:44.916576 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:44.916826 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "5f5e173144b047998f485728e7395258"
format_stamp: "Formatted at 2026-09-26 18:02:44 on dist-test-slave-5fqf"
I20260926 18:02:44.917088 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:44.930830 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:44.931720 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:44.962990 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:36873
I20260926 18:02:44.963073 10306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:36873 every 8 connection(s)
I20260926 18:02:44.966377 10307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:44.975378 10307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258: Bootstrap starting.
I20260926 18:02:44.979063 10307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:44.982496 10307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258: No bootstrap required, opened a new log
I20260926 18:02:44.984143 10307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER }
I20260926 18:02:44.984514 10307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:44.984687 10307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f5e173144b047998f485728e7395258, State: Initialized, Role: FOLLOWER
I20260926 18:02:44.985127 10307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER }
I20260926 18:02:44.985491 10307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:44.985683 10307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:44.985872 10307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:44.989658 10307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER }
I20260926 18:02:44.990067 10307 leader_election.cc:302] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f5e173144b047998f485728e7395258; no voters:
I20260926 18:02:44.991003 10307 leader_election.cc:288] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:44.991261 10310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:44.992482 10310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 1 LEADER]: Becoming Leader. State: Replica: 5f5e173144b047998f485728e7395258, State: Running, Role: LEADER
I20260926 18:02:44.993142 10310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER }
I20260926 18:02:44.993628 10307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:44.995563 10312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f5e173144b047998f485728e7395258. Latest consensus state: current_term: 1 leader_uuid: "5f5e173144b047998f485728e7395258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER } }
I20260926 18:02:44.995537 10311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f5e173144b047998f485728e7395258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5e173144b047998f485728e7395258" member_type: VOTER } }
I20260926 18:02:44.996167 10312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:44.996224 10311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:45.003749 10315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:45.008013 10315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:45.009001 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:45.015362 10315 catalog_manager.cc:1347] Generated new cluster ID: 644dd9be3c52439ba3b5a6ffd2b4c0b4
I20260926 18:02:45.015599 10315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:45.057416 10315 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:45.058919 10315 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:45.070569 10315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258: Generated new TSK 0
I20260926 18:02:45.071103 10315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:45.139119 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:45.144239 10328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.145015 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.146651 10331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:45.147868 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:45.148584 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:45.148761 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:45.148896 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445765148883 us; error 0 us; skew 500 ppm
I20260926 18:02:45.149355 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:45.151414 6506 webserver.cc:460] Webserver started at http://127.6.90.129:40605/ using document root <none> and password file <none>
I20260926 18:02:45.151831 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:45.151984 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:45.152202 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:45.153195 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "eb7f5d55fba1489e8c4ae10d3098f947"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.157061 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 18:02:45.159821 10336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.160461 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:45.160710 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "eb7f5d55fba1489e8c4ae10d3098f947"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.160945 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:45.171636 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:45.172479 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:45.173915 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:45.175913 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:45.176077 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.176308 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:45.176465 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 18:02:45.210160 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:45127
I20260926 18:02:45.210237 10398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:45127 every 8 connection(s)
I20260926 18:02:45.214154 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:45.220782 10403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.221887 10404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.224532 10406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:45.225625 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:45.226536 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:45.226778 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:45.226931 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445765226917 us; error 0 us; skew 500 ppm
I20260926 18:02:45.227388 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:45.227711 10399 heartbeater.cc:347] Connected to a master server at 127.6.90.190:36873
I20260926 18:02:45.228051 10399 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:45.228690 10399 heartbeater.cc:511] Master 127.6.90.190:36873 requested a full tablet report, sending...
I20260926 18:02:45.229887 6506 webserver.cc:460] Webserver started at http://127.6.90.130:34515/ using document root <none> and password file <none>
I20260926 18:02:45.230334 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:45.230288 10272 ts_manager.cc:194] Registered new tserver with Master: eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127)
I20260926 18:02:45.230638 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:45.230897 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:45.231842 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "5689405c499e48c3aa70add74d1afcff"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.232169 10272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55578
I20260926 18:02:45.235805 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 18:02:45.238701 10411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.239339 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 18:02:45.239580 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "5689405c499e48c3aa70add74d1afcff"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.239815 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:45.254177 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:45.254999 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:45.256184 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:45.258128 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:45.258298 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.258502 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:45.258633 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.291270 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:35675
I20260926 18:02:45.291352 10473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:35675 every 8 connection(s)
I20260926 18:02:45.295511 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:45.302341 10477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.302977 10478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:45.305512 10480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:45.306274 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:45.307071 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:45.307310 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:45.307502 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445765307479 us; error 0 us; skew 500 ppm
I20260926 18:02:45.307994 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:45.308286 10474 heartbeater.cc:347] Connected to a master server at 127.6.90.190:36873
I20260926 18:02:45.308653 10474 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:45.309379 10474 heartbeater.cc:511] Master 127.6.90.190:36873 requested a full tablet report, sending...
I20260926 18:02:45.310321 6506 webserver.cc:460] Webserver started at http://127.6.90.131:39319/ using document root <none> and password file <none>
I20260926 18:02:45.310798 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:45.310961 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:45.311199 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:45.311169 10272 ts_manager.cc:194] Registered new tserver with Master: 5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675)
I20260926 18:02:45.312314 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-2-root/instance:
uuid: "39ee996061104754acdc6df2ad4fdd63"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.312836 10272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55580
I20260926 18:02:45.316322 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 18:02:45.319581 10485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.320281 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260926 18:02:45.320555 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-2-root
uuid: "39ee996061104754acdc6df2ad4fdd63"
format_stamp: "Formatted at 2026-09-26 18:02:45 on dist-test-slave-5fqf"
I20260926 18:02:45.320775 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:45.334952 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:45.335767 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:45.336936 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:45.338863 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:45.339064 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.339264 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:45.339389 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:45.373205 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.131:35101
I20260926 18:02:45.373298 10547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.131:35101 every 8 connection(s)
I20260926 18:02:45.386404 10548 heartbeater.cc:347] Connected to a master server at 127.6.90.190:36873
I20260926 18:02:45.386809 10548 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:45.387425 10548 heartbeater.cc:511] Master 127.6.90.190:36873 requested a full tablet report, sending...
I20260926 18:02:45.389287 10272 ts_manager.cc:194] Registered new tserver with Master: 39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101)
I20260926 18:02:45.390362 6506 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01435012s
I20260926 18:02:45.390733 10272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55588
I20260926 18:02:45.409406 6506 test_util.cc:268] Using random seed: -1546241429
I20260926 18:02:45.429718 10272 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55604:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 18:02:45.471670 10513 tablet_service.cc:1467] Processing CreateTablet for tablet 43268c8e215442849d2d3428f3aaf66e (DEFAULT_TABLE table=test-workload [id=9a0ffdc2ec714e9e9148ea4caee3d748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:45.472911 10513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43268c8e215442849d2d3428f3aaf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:45.476020 10439 tablet_service.cc:1467] Processing CreateTablet for tablet 43268c8e215442849d2d3428f3aaf66e (DEFAULT_TABLE table=test-workload [id=9a0ffdc2ec714e9e9148ea4caee3d748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:45.477190 10439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43268c8e215442849d2d3428f3aaf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:45.481953 10364 tablet_service.cc:1467] Processing CreateTablet for tablet 43268c8e215442849d2d3428f3aaf66e (DEFAULT_TABLE table=test-workload [id=9a0ffdc2ec714e9e9148ea4caee3d748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:45.483129 10364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43268c8e215442849d2d3428f3aaf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:45.487406 10568 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Bootstrap starting.
I20260926 18:02:45.489384 10569 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: Bootstrap starting.
I20260926 18:02:45.493413 10568 tablet_bootstrap.cc:654] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:45.495138 10569 tablet_bootstrap.cc:654] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:45.497926 10571 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Bootstrap starting.
I20260926 18:02:45.498847 10568 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: No bootstrap required, opened a new log
I20260926 18:02:45.499212 10568 ts_tablet_manager.cc:1397] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260926 18:02:45.500957 10569 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: No bootstrap required, opened a new log
I20260926 18:02:45.501530 10569 ts_tablet_manager.cc:1397] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260926 18:02:45.501668 10568 raft_consensus.cc:348] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.502450 10568 raft_consensus.cc:374] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:45.502732 10568 raft_consensus.cc:729] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ee996061104754acdc6df2ad4fdd63, State: Initialized, Role: FOLLOWER
I20260926 18:02:45.503873 10571 tablet_bootstrap.cc:654] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:45.503474 10568 consensus_queue.cc:260] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.504676 10569 raft_consensus.cc:348] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.505468 10569 raft_consensus.cc:374] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:45.505796 10569 raft_consensus.cc:729] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5689405c499e48c3aa70add74d1afcff, State: Initialized, Role: FOLLOWER
I20260926 18:02:45.506524 10569 consensus_queue.cc:260] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.511288 10548 heartbeater.cc:503] Master 127.6.90.190:36873 was elected leader, sending a full tablet report...
I20260926 18:02:45.511770 10568 ts_tablet_manager.cc:1428] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Time spent starting tablet: real 0.012s user 0.001s sys 0.008s
I20260926 18:02:45.512029 10474 heartbeater.cc:503] Master 127.6.90.190:36873 was elected leader, sending a full tablet report...
I20260926 18:02:45.513365 10571 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: No bootstrap required, opened a new log
I20260926 18:02:45.513398 10569 ts_tablet_manager.cc:1428] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260926 18:02:45.513973 10571 ts_tablet_manager.cc:1397] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.009s
I20260926 18:02:45.516669 10571 raft_consensus.cc:348] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.517267 10571 raft_consensus.cc:374] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:45.517555 10571 raft_consensus.cc:729] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7f5d55fba1489e8c4ae10d3098f947, State: Initialized, Role: FOLLOWER
I20260926 18:02:45.518231 10571 consensus_queue.cc:260] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.520776 10399 heartbeater.cc:503] Master 127.6.90.190:36873 was elected leader, sending a full tablet report...
I20260926 18:02:45.521344 10571 ts_tablet_manager.cc:1428] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 18:02:45.736889 10573 raft_consensus.cc:482] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:45.737324 10573 raft_consensus.cc:504] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.739135 10573 leader_election.cc:288] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675), eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127)
I20260926 18:02:45.749914 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43268c8e215442849d2d3428f3aaf66e" candidate_uuid: "39ee996061104754acdc6df2ad4fdd63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5689405c499e48c3aa70add74d1afcff" is_pre_election: true
I20260926 18:02:45.750598 10449 raft_consensus.cc:2454] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ee996061104754acdc6df2ad4fdd63 in term 0.
I20260926 18:02:45.751354 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43268c8e215442849d2d3428f3aaf66e" candidate_uuid: "39ee996061104754acdc6df2ad4fdd63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" is_pre_election: true
I20260926 18:02:45.751415 10489 leader_election.cc:302] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ee996061104754acdc6df2ad4fdd63, 5689405c499e48c3aa70add74d1afcff; no voters:
I20260926 18:02:45.751906 10374 raft_consensus.cc:2454] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ee996061104754acdc6df2ad4fdd63 in term 0.
I20260926 18:02:45.752159 10573 raft_consensus.cc:2781] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:45.752566 10573 raft_consensus.cc:482] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:45.752846 10573 raft_consensus.cc:3037] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:45.756897 10573 raft_consensus.cc:504] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.758193 10573 leader_election.cc:288] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [CANDIDATE]: Term 1 election: Requested vote from peers 5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675), eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127)
I20260926 18:02:45.758934 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43268c8e215442849d2d3428f3aaf66e" candidate_uuid: "39ee996061104754acdc6df2ad4fdd63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5689405c499e48c3aa70add74d1afcff"
I20260926 18:02:45.759047 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43268c8e215442849d2d3428f3aaf66e" candidate_uuid: "39ee996061104754acdc6df2ad4fdd63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7f5d55fba1489e8c4ae10d3098f947"
I20260926 18:02:45.759357 10449 raft_consensus.cc:3037] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:45.759510 10374 raft_consensus.cc:3037] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:45.763628 10449 raft_consensus.cc:2454] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ee996061104754acdc6df2ad4fdd63 in term 1.
I20260926 18:02:45.763799 10374 raft_consensus.cc:2454] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ee996061104754acdc6df2ad4fdd63 in term 1.
I20260926 18:02:45.764504 10489 leader_election.cc:302] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ee996061104754acdc6df2ad4fdd63, 5689405c499e48c3aa70add74d1afcff; no voters:
I20260926 18:02:45.765090 10573 raft_consensus.cc:2781] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:45.765872 10573 raft_consensus.cc:686] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 LEADER]: Becoming Leader. State: Replica: 39ee996061104754acdc6df2ad4fdd63, State: Running, Role: LEADER
I20260926 18:02:45.767113 10573 consensus_queue.cc:237] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:45.774104 10269 catalog_manager.cc:5495] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39ee996061104754acdc6df2ad4fdd63 (127.6.90.131). New cstate: current_term: 1 leader_uuid: "39ee996061104754acdc6df2ad4fdd63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } }
I20260926 18:02:46.056690 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:46.059580 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:46.061190 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 43268c8e215442849d2d3428f3aaf66e of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675): RUNNING
eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): RUNNING
39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101): RUNNING [LEADER]
All reported replicas are:
A = 5689405c499e48c3aa70add74d1afcff
B = eb7f5d55fba1489e8c4ae10d3098f947
C = 39ee996061104754acdc6df2ad4fdd63
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:46.254956 10573 consensus_queue.cc:1035] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 18:02:46.261349 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:46.267445 10573 consensus_queue.cc:1035] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:46.604470 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:46.613787 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:46.621346 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260926 18:02:46.793830 6506 log_verifier.cc:126] Checking tablet 43268c8e215442849d2d3428f3aaf66e
I20260926 18:02:46.847927 6506 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 43268c8e215442849d2d3428f3aaf66e
I20260926 18:02:46.848510 6506 tablet_replica.cc:330] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: stopping tablet replica
I20260926 18:02:46.848996 6506 raft_consensus.cc:2229] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:46.849404 6506 raft_consensus.cc:2258] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:47.064142 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:47.064325 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:47.064977 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 43268c8e215442849d2d3428f3aaf66e of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675): RUNNING
eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101): RUNNING [LEADER]
All reported replicas are:
A = 5689405c499e48c3aa70add74d1afcff
B = eb7f5d55fba1489e8c4ae10d3098f947
C = 39ee996061104754acdc6df2ad4fdd63
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | [config not available] | | |
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:47.247249 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
W20260926 18:02:47.277071 10487 consensus_peers.cc:487] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 -> Peer eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): Couldn't send request to peer eb7f5d55fba1489e8c4ae10d3098f947. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:47.278017 10578 raft_consensus.cc:998] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Attempting to remove follower eb7f5d55fba1489e8c4ae10d3098f947 from the Raft config. Reason: The tablet replica hosted on peer eb7f5d55fba1489e8c4ae10d3098f947 has failed
I20260926 18:02:47.279327 10578 consensus_queue.cc:237] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } }
I20260926 18:02:47.283464 10449 raft_consensus.cc:1261] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Refusing update from remote peer 39ee996061104754acdc6df2ad4fdd63: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 18:02:47.284513 10664 consensus_queue.cc:1035] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:47.289695 10578 raft_consensus.cc:2932] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } }
I20260926 18:02:47.291045 10449 raft_consensus.cc:2932] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } }
I20260926 18:02:47.300304 10268 catalog_manager.cc:5495] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 reported cstate change: config changed from index -1 to 2, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) evicted. New cstate: current_term: 1 leader_uuid: "39ee996061104754acdc6df2ad4fdd63" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } }
I20260926 18:02:47.311228 10364 tablet_service.cc:1514] Processing DeleteTablet for tablet 43268c8e215442849d2d3428f3aaf66e with delete_type TABLET_DATA_TOMBSTONED (TS eb7f5d55fba1489e8c4ae10d3098f947 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:51190
I20260926 18:02:47.313373 10672 ts_tablet_manager.cc:1905] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260926 18:02:47.313560 10523 consensus_queue.cc:237] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } }
I20260926 18:02:47.317386 10449 raft_consensus.cc:1261] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Refusing update from remote peer 39ee996061104754acdc6df2ad4fdd63: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260926 18:02:47.318172 10487 consensus_peers.cc:487] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 -> Peer eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): Couldn't send request to peer eb7f5d55fba1489e8c4ae10d3098f947. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:47.318679 10578 consensus_queue.cc:1035] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:47.323879 10664 raft_consensus.cc:2932] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } }
I20260926 18:02:47.325650 10672 ts_tablet_manager.cc:1918] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260926 18:02:47.326018 10672 log.cc:1196] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1790445738975945-6506-0/minicluster-data/ts-0-root/wals/43268c8e215442849d2d3428f3aaf66e
I20260926 18:02:47.325349 10449 raft_consensus.cc:2932] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } }
I20260926 18:02:47.327361 10257 catalog_manager.cc:4841] TS eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): tablet 43268c8e215442849d2d3428f3aaf66e (table test-workload [id=9a0ffdc2ec714e9e9148ea4caee3d748]) successfully deleted
I20260926 18:02:47.329859 10259 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet 43268c8e215442849d2d3428f3aaf66e with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260926 18:02:47.332624 10268 catalog_manager.cc:5495] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 reported cstate change: config changed from index 2 to 3, VOTER eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129) added. New cstate: current_term: 1 leader_uuid: "39ee996061104754acdc6df2ad4fdd63" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } } }
I20260926 18:02:47.597270 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:47.602181 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:47.602296 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:47.753319 10699 ts_tablet_manager.cc:927] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Initiating tablet copy from peer 39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101)
I20260926 18:02:47.755152 10699 tablet_copy_client.cc:316] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet copy: Beginning tablet copy session from remote peer at address 127.6.90.131:35101
I20260926 18:02:47.771848 10533 tablet_copy_service.cc:140] P 39ee996061104754acdc6df2ad4fdd63: Received BeginTabletCopySession request for tablet 43268c8e215442849d2d3428f3aaf66e from peer eb7f5d55fba1489e8c4ae10d3098f947 ({username='slave'} at 127.0.0.1:50742)
I20260926 18:02:47.772287 10533 tablet_copy_service.cc:161] P 39ee996061104754acdc6df2ad4fdd63: Beginning new tablet copy session on tablet 43268c8e215442849d2d3428f3aaf66e from peer eb7f5d55fba1489e8c4ae10d3098f947 at {username='slave'} at 127.0.0.1:50742: session id = eb7f5d55fba1489e8c4ae10d3098f947-43268c8e215442849d2d3428f3aaf66e
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 43268c8e215442849d2d3428f3aaf66e of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675): RUNNING
39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101): RUNNING [LEADER]
eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): not running
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Deleted tablet blocks from disk
All reported replicas are:
A = 5689405c499e48c3aa70add74d1afcff
B = 39ee996061104754acdc6df2ad4fdd63
C = eb7f5d55fba1489e8c4ae10d3098f947
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 3 | Yes
B | A B* C | 1 | 3 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
I20260926 18:02:47.777958 10533 tablet_copy_source_session.cc:205] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: Tablet Copy: opened 0 blocks and 1 log segments
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:47.780194 10699 ts_tablet_manager.cc:1905] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260926 18:02:47.782178 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:47.790855 10699 ts_tablet_manager.cc:1918] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260926 18:02:47.791349 10699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43268c8e215442849d2d3428f3aaf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:47.795944 10699 tablet_copy_client.cc:791] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet copy: Starting download of 0 data blocks...
I20260926 18:02:47.796378 10699 tablet_copy_client.cc:655] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet copy: Starting download of 1 WAL segments...
I20260926 18:02:47.798974 10699 tablet_copy_client.cc:531] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 18:02:47.803841 10699 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Bootstrap starting.
I20260926 18:02:47.817087 10699 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:47.817631 10699 tablet_bootstrap.cc:492] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Bootstrap complete.
I20260926 18:02:47.818043 10699 ts_tablet_manager.cc:1397] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.004s
I20260926 18:02:47.819720 10699 raft_consensus.cc:348] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } }
I20260926 18:02:47.820184 10699 raft_consensus.cc:729] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7f5d55fba1489e8c4ae10d3098f947, State: Initialized, Role: FOLLOWER
I20260926 18:02:47.820706 10699 consensus_queue.cc:260] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5689405c499e48c3aa70add74d1afcff" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 35675 } } peers { permanent_uuid: "39ee996061104754acdc6df2ad4fdd63" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 35101 } } peers { permanent_uuid: "eb7f5d55fba1489e8c4ae10d3098f947" member_type: VOTER last_known_addr { host: "127.6.90.129" port: 45127 } }
I20260926 18:02:47.822427 10699 ts_tablet_manager.cc:1428] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260926 18:02:47.823858 10533 tablet_copy_service.cc:342] P 39ee996061104754acdc6df2ad4fdd63: Request end of tablet copy session eb7f5d55fba1489e8c4ae10d3098f947-43268c8e215442849d2d3428f3aaf66e received from {username='slave'} at 127.0.0.1:50742
I20260926 18:02:47.824218 10533 tablet_copy_service.cc:433] P 39ee996061104754acdc6df2ad4fdd63: ending tablet copy session eb7f5d55fba1489e8c4ae10d3098f947-43268c8e215442849d2d3428f3aaf66e on tablet 43268c8e215442849d2d3428f3aaf66e with peer eb7f5d55fba1489e8c4ae10d3098f947
I20260926 18:02:48.197376 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:48.197584 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:48.203759 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:48.273734 10374 raft_consensus.cc:1203] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3] Dedup: 1.3->[]
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 43268c8e215442849d2d3428f3aaf66e of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
5689405c499e48c3aa70add74d1afcff (127.6.90.130:35675): RUNNING
39ee996061104754acdc6df2ad4fdd63 (127.6.90.131:35101): RUNNING [LEADER]
eb7f5d55fba1489e8c4ae10d3098f947 (127.6.90.129:45127): RUNNING
All reported replicas are:
A = 5689405c499e48c3aa70add74d1afcff
B = 39ee996061104754acdc6df2ad4fdd63
C = eb7f5d55fba1489e8c4ae10d3098f947
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 3 | Yes
B | A B* C | 1 | 3 | Yes
C | A B C | 1 | 3 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:48.380146 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:48.918906 10513 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:48.924924 10439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:48.928567 10364 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
5f5e173144b047998f485728e7395258 | 127.6.90.190:36873 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
39ee996061104754acdc6df2ad4fdd63 | 127.6.90.131:35101 | HEALTHY | <none> | 1 | 0
5689405c499e48c3aa70add74d1afcff | 127.6.90.130:35675 | HEALTHY | <none> | 0 | 0
eb7f5d55fba1489e8c4ae10d3098f947 | 127.6.90.129:45127 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260926 18:02:49.101943 6506 log_verifier.cc:126] Checking tablet 43268c8e215442849d2d3428f3aaf66e
I20260926 18:02:49.154495 6506 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 43268c8e215442849d2d3428f3aaf66e
I20260926 18:02:49.157095 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:49.174705 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:49.175261 6506 tablet_replica.cc:330] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947: stopping tablet replica
I20260926 18:02:49.175735 6506 raft_consensus.cc:2229] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:49.176072 6506 raft_consensus.cc:2258] T 43268c8e215442849d2d3428f3aaf66e P eb7f5d55fba1489e8c4ae10d3098f947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:49.194283 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:49.202730 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:49.217821 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:49.218425 6506 tablet_replica.cc:330] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff: stopping tablet replica
I20260926 18:02:49.219009 6506 raft_consensus.cc:2229] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:49.219498 6506 raft_consensus.cc:2258] T 43268c8e215442849d2d3428f3aaf66e P 5689405c499e48c3aa70add74d1afcff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:49.237715 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:49.245607 6506 tablet_server.cc:178] TabletServer@127.6.90.131:0 shutting down...
I20260926 18:02:49.261690 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:49.262226 6506 tablet_replica.cc:330] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63: stopping tablet replica
I20260926 18:02:49.262682 6506 raft_consensus.cc:2229] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:49.263396 6506 raft_consensus.cc:2258] T 43268c8e215442849d2d3428f3aaf66e P 39ee996061104754acdc6df2ad4fdd63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:49.291286 6506 tablet_server.cc:194] TabletServer@127.6.90.131:0 shutdown complete.
I20260926 18:02:49.306123 6506 master.cc:537] Master@127.6.90.190:36873 shutting down...
I20260926 18:02:49.325129 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:49.325928 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:49.326316 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5f5e173144b047998f485728e7395258: stopping tablet replica
I20260926 18:02:49.345913 6506 master.cc:552] Master@127.6.90.190:36873 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (4487 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260926 18:02:49.383189 6506 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.90.190:37829
I20260926 18:02:49.384198 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:49.389439 10762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.389883 10763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.393313 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:49.393906 10765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.394626 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:49.394806 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:49.394929 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445769394921 us; error 0 us; skew 500 ppm
I20260926 18:02:49.395335 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:49.397567 6506 webserver.cc:460] Webserver started at http://127.6.90.190:36539/ using document root <none> and password file <none>
I20260926 18:02:49.398005 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:49.398213 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:49.398540 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:49.400079 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/master-0-root/instance:
uuid: "44636d5fbfa948e88f08cc2708b0d9f5"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.404780 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 18:02:49.407811 10770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.408563 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 18:02:49.408782 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/master-0-root
uuid: "44636d5fbfa948e88f08cc2708b0d9f5"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.409003 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:49.426903 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:49.427975 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:49.462150 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.190:37829
I20260926 18:02:49.462258 10821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.190:37829 every 8 connection(s)
I20260926 18:02:49.465688 10822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:49.476456 10822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5: Bootstrap starting.
I20260926 18:02:49.480676 10822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:49.484504 10822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5: No bootstrap required, opened a new log
I20260926 18:02:49.486397 10822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER }
I20260926 18:02:49.486858 10822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:49.487058 10822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44636d5fbfa948e88f08cc2708b0d9f5, State: Initialized, Role: FOLLOWER
I20260926 18:02:49.487598 10822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER }
I20260926 18:02:49.488029 10822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 18:02:49.488241 10822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 18:02:49.488509 10822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:49.492517 10822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER }
I20260926 18:02:49.492962 10822 leader_election.cc:302] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44636d5fbfa948e88f08cc2708b0d9f5; no voters:
I20260926 18:02:49.493965 10822 leader_election.cc:288] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 18:02:49.494318 10825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:49.495522 10825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 44636d5fbfa948e88f08cc2708b0d9f5, State: Running, Role: LEADER
I20260926 18:02:49.496045 10825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER }
I20260926 18:02:49.496622 10822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 18:02:49.498773 10826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER } }
I20260926 18:02:49.498768 10827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44636d5fbfa948e88f08cc2708b0d9f5. Latest consensus state: current_term: 1 leader_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44636d5fbfa948e88f08cc2708b0d9f5" member_type: VOTER } }
I20260926 18:02:49.499428 10826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:49.499575 10827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [sys.catalog]: This master's current role is: LEADER
I20260926 18:02:49.527771 10830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 18:02:49.532495 10830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 18:02:49.533458 6506 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 18:02:49.540310 10830 catalog_manager.cc:1347] Generated new cluster ID: 8934bd5a3ed0472f88e69ce47b9f87d6
I20260926 18:02:49.540557 10830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 18:02:49.559191 10830 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 18:02:49.560798 10830 catalog_manager.cc:1504] Loading token signing keys...
I20260926 18:02:49.574594 10830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5: Generated new TSK 0
I20260926 18:02:49.575166 10830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 18:02:49.599431 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:49.604910 10843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.605741 10844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.607635 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:49.607877 10846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.608768 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:49.608953 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:49.609109 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445769609093 us; error 0 us; skew 500 ppm
I20260926 18:02:49.609598 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:49.611891 6506 webserver.cc:460] Webserver started at http://127.6.90.129:41473/ using document root <none> and password file <none>
I20260926 18:02:49.612394 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:49.612566 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:49.612800 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:49.613840 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-0-root/instance:
uuid: "07d8fe4d5993476fa862374ffa11b2b4"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.617848 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 18:02:49.620833 10851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.621531 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260926 18:02:49.621754 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-0-root
uuid: "07d8fe4d5993476fa862374ffa11b2b4"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.621979 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:49.649528 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:49.650605 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:49.651875 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:49.653966 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:49.654151 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.654358 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:49.654507 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.691550 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.129:46411
I20260926 18:02:49.691645 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.129:46411 every 8 connection(s)
I20260926 18:02:49.695794 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:49.702837 10918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.703480 10919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.705621 10921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.705796 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:49.706641 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:49.706830 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:49.706990 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445769706973 us; error 0 us; skew 500 ppm
I20260926 18:02:49.707475 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:49.710005 6506 webserver.cc:460] Webserver started at http://127.6.90.130:46403/ using document root <none> and password file <none>
I20260926 18:02:49.710546 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:49.710765 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:49.710825 10914 heartbeater.cc:347] Connected to a master server at 127.6.90.190:37829
I20260926 18:02:49.711117 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:49.711213 10914 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:49.711984 10914 heartbeater.cc:511] Master 127.6.90.190:37829 requested a full tablet report, sending...
I20260926 18:02:49.713881 10787 ts_manager.cc:194] Registered new tserver with Master: 07d8fe4d5993476fa862374ffa11b2b4 (127.6.90.129:46411)
I20260926 18:02:49.713932 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-1-root/instance:
uuid: "0b81d04cd3474908a47f38ee64f48d1d"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.715773 10787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48520
I20260926 18:02:49.719004 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 18:02:49.722194 10926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.722872 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 18:02:49.723127 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-1-root
uuid: "0b81d04cd3474908a47f38ee64f48d1d"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.723371 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:49.732894 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:49.733946 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:49.735188 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:49.737210 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:49.737372 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.737579 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:49.737725 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.774112 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.130:33107
I20260926 18:02:49.774214 10988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.130:33107 every 8 connection(s)
I20260926 18:02:49.778332 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:49.785092 10992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.787191 10993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.788460 6506 server_base.cc:1034] running on GCE node
W20260926 18:02:49.789021 10995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.789819 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 18:02:49.790017 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:49.790192 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445769790175 us; error 0 us; skew 500 ppm
I20260926 18:02:49.790719 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:49.791395 10989 heartbeater.cc:347] Connected to a master server at 127.6.90.190:37829
I20260926 18:02:49.791810 10989 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:49.792609 10989 heartbeater.cc:511] Master 127.6.90.190:37829 requested a full tablet report, sending...
I20260926 18:02:49.793334 6506 webserver.cc:460] Webserver started at http://127.6.90.131:33817/ using document root <none> and password file <none>
I20260926 18:02:49.793838 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:49.794027 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:49.794266 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:49.794487 10787 ts_manager.cc:194] Registered new tserver with Master: 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107)
I20260926 18:02:49.795399 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-2-root/instance:
uuid: "da04736c877f463084a56916cfee8975"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.795989 10787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33974
I20260926 18:02:49.799688 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 18:02:49.802816 11000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.803575 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 18:02:49.803902 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-2-root
uuid: "da04736c877f463084a56916cfee8975"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.804183 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:49.823835 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:49.825145 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:49.826529 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:49.828662 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:49.828830 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.829052 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:49.829185 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.865432 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.131:46253
I20260926 18:02:49.865540 11062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.131:46253 every 8 connection(s)
I20260926 18:02:49.869727 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 18:02:49.877480 11066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.880300 11067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 18:02:49.880954 11069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 18:02:49.883011 6506 server_base.cc:1034] running on GCE node
I20260926 18:02:49.883661 11063 heartbeater.cc:347] Connected to a master server at 127.6.90.190:37829
I20260926 18:02:49.883955 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260926 18:02:49.884049 11063 heartbeater.cc:464] Registering TS with master...
W20260926 18:02:49.884238 6506 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 18:02:49.884467 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790445769884449 us; error 0 us; skew 500 ppm
I20260926 18:02:49.884864 11063 heartbeater.cc:511] Master 127.6.90.190:37829 requested a full tablet report, sending...
I20260926 18:02:49.885187 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 18:02:49.886993 10787 ts_manager.cc:194] Registered new tserver with Master: da04736c877f463084a56916cfee8975 (127.6.90.131:46253)
I20260926 18:02:49.887881 6506 webserver.cc:460] Webserver started at http://127.6.90.132:44935/ using document root <none> and password file <none>
I20260926 18:02:49.888509 6506 fs_manager.cc:359] Metadata directory not provided
I20260926 18:02:49.888737 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 18:02:49.889045 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 18:02:49.889125 10787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33980
I20260926 18:02:49.890509 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root/instance:
uuid: "7a1c9473d9234bca9c36f66cab316deb"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.894979 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 18:02:49.898075 11074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.898737 6506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 18:02:49.898984 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root
uuid: "7a1c9473d9234bca9c36f66cab316deb"
format_stamp: "Formatted at 2026-09-26 18:02:49 on dist-test-slave-5fqf"
I20260926 18:02:49.899223 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 18:02:49.931355 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 18:02:49.932577 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 18:02:49.933754 6506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 18:02:49.935822 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 18:02:49.935988 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.936209 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 18:02:49.936367 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 18:02:49.972463 6506 rpc_server.cc:307] RPC server started. Bound to: 127.6.90.132:44363
I20260926 18:02:49.972589 11136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.90.132:44363 every 8 connection(s)
I20260926 18:02:49.986546 11137 heartbeater.cc:347] Connected to a master server at 127.6.90.190:37829
I20260926 18:02:49.986892 11137 heartbeater.cc:464] Registering TS with master...
I20260926 18:02:49.987548 11137 heartbeater.cc:511] Master 127.6.90.190:37829 requested a full tablet report, sending...
I20260926 18:02:49.989180 10787 ts_manager.cc:194] Registered new tserver with Master: 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363)
I20260926 18:02:49.989835 6506 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.014519059s
I20260926 18:02:49.990836 10787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33990
I20260926 18:02:50.010600 6506 test_util.cc:268] Using random seed: -1541640237
I20260926 18:02:50.031350 10787 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34008:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 18:02:50.067526 11028 tablet_service.cc:1467] Processing CreateTablet for tablet 57c49cb6f86f4bf791934acb0c8886c2 (DEFAULT_TABLE table=test-workload [id=4dacbc06f9924943bdb9a2f4a575783f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:50.068845 11028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c49cb6f86f4bf791934acb0c8886c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:50.080590 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 57c49cb6f86f4bf791934acb0c8886c2 (DEFAULT_TABLE table=test-workload [id=4dacbc06f9924943bdb9a2f4a575783f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:50.081699 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c49cb6f86f4bf791934acb0c8886c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:50.084558 11157 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: Bootstrap starting.
I20260926 18:02:50.084406 11102 tablet_service.cc:1467] Processing CreateTablet for tablet 57c49cb6f86f4bf791934acb0c8886c2 (DEFAULT_TABLE table=test-workload [id=4dacbc06f9924943bdb9a2f4a575783f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 18:02:50.085485 11102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c49cb6f86f4bf791934acb0c8886c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:50.090754 11157 tablet_bootstrap.cc:654] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:50.095983 11157 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: No bootstrap required, opened a new log
I20260926 18:02:50.096100 11159 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: Bootstrap starting.
I20260926 18:02:50.096419 11157 ts_tablet_manager.cc:1397] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260926 18:02:50.097853 11160 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Bootstrap starting.
I20260926 18:02:50.099290 11157 raft_consensus.cc:348] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.100056 11157 raft_consensus.cc:374] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:50.100358 11157 raft_consensus.cc:729] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da04736c877f463084a56916cfee8975, State: Initialized, Role: FOLLOWER
I20260926 18:02:50.101497 11159 tablet_bootstrap.cc:654] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:50.101011 11157 consensus_queue.cc:260] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.104207 11160 tablet_bootstrap.cc:654] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Neither blocks nor log segments found. Creating new log.
I20260926 18:02:50.105540 11063 heartbeater.cc:503] Master 127.6.90.190:37829 was elected leader, sending a full tablet report...
I20260926 18:02:50.105888 11157 ts_tablet_manager.cc:1428] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260926 18:02:50.109078 11160 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: No bootstrap required, opened a new log
I20260926 18:02:50.109532 11160 ts_tablet_manager.cc:1397] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260926 18:02:50.112092 11160 raft_consensus.cc:348] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.112762 11159 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: No bootstrap required, opened a new log
I20260926 18:02:50.112835 11160 raft_consensus.cc:374] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:50.113257 11159 ts_tablet_manager.cc:1397] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260926 18:02:50.113204 11160 raft_consensus.cc:729] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1c9473d9234bca9c36f66cab316deb, State: Initialized, Role: FOLLOWER
I20260926 18:02:50.114035 11160 consensus_queue.cc:260] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.116278 11137 heartbeater.cc:503] Master 127.6.90.190:37829 was elected leader, sending a full tablet report...
I20260926 18:02:50.115948 11159 raft_consensus.cc:348] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.116714 11159 raft_consensus.cc:374] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 18:02:50.116777 11160 ts_tablet_manager.cc:1428] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260926 18:02:50.117098 11159 raft_consensus.cc:729] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b81d04cd3474908a47f38ee64f48d1d, State: Initialized, Role: FOLLOWER
I20260926 18:02:50.117749 11159 consensus_queue.cc:260] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.119711 10989 heartbeater.cc:503] Master 127.6.90.190:37829 was elected leader, sending a full tablet report...
I20260926 18:02:50.119997 11159 ts_tablet_manager.cc:1428] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 18:02:50.264223 11164 raft_consensus.cc:482] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 18:02:50.264645 11164 raft_consensus.cc:504] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.266304 11164 leader_election.cc:288] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da04736c877f463084a56916cfee8975 (127.6.90.131:46253), 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107)
I20260926 18:02:50.279341 10964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "7a1c9473d9234bca9c36f66cab316deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b81d04cd3474908a47f38ee64f48d1d" is_pre_election: true
I20260926 18:02:50.279543 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "7a1c9473d9234bca9c36f66cab316deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da04736c877f463084a56916cfee8975" is_pre_election: true
I20260926 18:02:50.280141 10964 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1c9473d9234bca9c36f66cab316deb in term 0.
I20260926 18:02:50.280162 11038 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1c9473d9234bca9c36f66cab316deb in term 0.
I20260926 18:02:50.281368 11078 leader_election.cc:302] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b81d04cd3474908a47f38ee64f48d1d, 7a1c9473d9234bca9c36f66cab316deb; no voters:
I20260926 18:02:50.281993 11164 raft_consensus.cc:2781] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 18:02:50.282320 11164 raft_consensus.cc:482] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 18:02:50.282598 11164 raft_consensus.cc:3037] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:50.287528 11164 raft_consensus.cc:504] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.288915 11164 leader_election.cc:288] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [CANDIDATE]: Term 1 election: Requested vote from peers da04736c877f463084a56916cfee8975 (127.6.90.131:46253), 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107)
I20260926 18:02:50.289601 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "7a1c9473d9234bca9c36f66cab316deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da04736c877f463084a56916cfee8975"
I20260926 18:02:50.289794 10964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "7a1c9473d9234bca9c36f66cab316deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b81d04cd3474908a47f38ee64f48d1d"
I20260926 18:02:50.290165 11038 raft_consensus.cc:3037] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:50.290290 10964 raft_consensus.cc:3037] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 0 FOLLOWER]: Advancing to term 1
I20260926 18:02:50.297078 11038 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1c9473d9234bca9c36f66cab316deb in term 1.
I20260926 18:02:50.297081 10964 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1c9473d9234bca9c36f66cab316deb in term 1.
I20260926 18:02:50.298142 11078 leader_election.cc:302] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b81d04cd3474908a47f38ee64f48d1d, 7a1c9473d9234bca9c36f66cab316deb; no voters:
I20260926 18:02:50.298733 11164 raft_consensus.cc:2781] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 18:02:50.299495 11164 raft_consensus.cc:686] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 1 LEADER]: Becoming Leader. State: Replica: 7a1c9473d9234bca9c36f66cab316deb, State: Running, Role: LEADER
I20260926 18:02:50.300088 11164 consensus_queue.cc:237] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:50.306131 10786 catalog_manager.cc:5495] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132). New cstate: current_term: 1 leader_uuid: "7a1c9473d9234bca9c36f66cab316deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } health_report { overall_health: HEALTHY } } }
I20260926 18:02:50.583125 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:50.585541 10954 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:50.588030 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:50.588239 11102 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:50.719043 10914 heartbeater.cc:503] Master 127.6.90.190:37829 was elected leader, sending a full tablet report...
I20260926 18:02:50.785324 11164 consensus_queue.cc:1035] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 18:02:50.808458 11164 consensus_queue.cc:1035] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 0 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 1 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 57c49cb6f86f4bf791934acb0c8886c2 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
da04736c877f463084a56916cfee8975 (127.6.90.131:46253): RUNNING
0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107): RUNNING
7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): RUNNING [LEADER]
All reported replicas are:
A = da04736c877f463084a56916cfee8975
B = 0b81d04cd3474908a47f38ee64f48d1d
C = 7a1c9473d9234bca9c36f66cab316deb
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:50.824127 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:51.172554 10954 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.177567 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.182268 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.186256 11102 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 0 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 1 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260926 18:02:51.389266 6506 log_verifier.cc:126] Checking tablet 57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:51.442618 6506 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:51.443065 6506 tablet_replica.cc:330] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: stopping tablet replica
I20260926 18:02:51.443480 6506 raft_consensus.cc:2229] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:51.444182 6506 raft_consensus.cc:2258] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:51.638986 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.648180 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.655203 10954 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:51.655889 11102 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 0 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 0 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 57c49cb6f86f4bf791934acb0c8886c2 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
da04736c877f463084a56916cfee8975 (127.6.90.131:46253): RUNNING
0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107): RUNNING
7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = da04736c877f463084a56916cfee8975
B = 0b81d04cd3474908a47f38ee64f48d1d
C = 7a1c9473d9234bca9c36f66cab316deb
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:51.856580 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:52.224551 10954 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:52.227171 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:52.229841 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:52.230700 11102 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 0 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 0 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 57c49cb6f86f4bf791934acb0c8886c2 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
da04736c877f463084a56916cfee8975 (127.6.90.131:46253): RUNNING
0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107): RUNNING
7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): not running [LEADER]
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = da04736c877f463084a56916cfee8975
B = 0b81d04cd3474908a47f38ee64f48d1d
C = 7a1c9473d9234bca9c36f66cab316deb
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 1 | -1 | Yes
B | A B C* | 1 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:52.436648 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:52.745190 11298 raft_consensus.cc:482] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a1c9473d9234bca9c36f66cab316deb)
I20260926 18:02:52.745661 11298 raft_consensus.cc:504] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:52.753619 11298 leader_election.cc:288] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da04736c877f463084a56916cfee8975 (127.6.90.131:46253), 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363)
I20260926 18:02:52.771150 11303 raft_consensus.cc:482] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a1c9473d9234bca9c36f66cab316deb)
I20260926 18:02:52.771745 11303 raft_consensus.cc:504] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:52.774108 11303 leader_election.cc:288] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107), 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363)
I20260926 18:02:52.780853 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "0b81d04cd3474908a47f38ee64f48d1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a1c9473d9234bca9c36f66cab316deb" is_pre_election: true
W20260926 18:02:52.782266 10930 leader_election.cc:341] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): Illegal state: cannot vote while shut down
I20260926 18:02:52.784099 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "0b81d04cd3474908a47f38ee64f48d1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da04736c877f463084a56916cfee8975" is_pre_election: true
I20260926 18:02:52.784737 11038 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b81d04cd3474908a47f38ee64f48d1d in term 1.
I20260926 18:02:52.785730 10927 leader_election.cc:302] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b81d04cd3474908a47f38ee64f48d1d, da04736c877f463084a56916cfee8975; no voters: 7a1c9473d9234bca9c36f66cab316deb
I20260926 18:02:52.786576 11298 raft_consensus.cc:2781] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 18:02:52.786926 11298 raft_consensus.cc:482] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a1c9473d9234bca9c36f66cab316deb)
I20260926 18:02:52.787249 11298 raft_consensus.cc:3037] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:52.793551 10964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "da04736c877f463084a56916cfee8975" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b81d04cd3474908a47f38ee64f48d1d" is_pre_election: true
I20260926 18:02:52.793860 11298 raft_consensus.cc:504] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:52.794979 10964 raft_consensus.cc:2379] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da04736c877f463084a56916cfee8975 in current term 2: Already voted for candidate 0b81d04cd3474908a47f38ee64f48d1d in this term.
I20260926 18:02:52.795708 11298 leader_election.cc:288] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 election: Requested vote from peers da04736c877f463084a56916cfee8975 (127.6.90.131:46253), 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363)
I20260926 18:02:52.797025 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "0b81d04cd3474908a47f38ee64f48d1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da04736c877f463084a56916cfee8975"
I20260926 18:02:52.798561 11038 raft_consensus.cc:3037] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 1 FOLLOWER]: Advancing to term 2
I20260926 18:02:52.800433 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "da04736c877f463084a56916cfee8975" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a1c9473d9234bca9c36f66cab316deb" is_pre_election: true
W20260926 18:02:52.801738 11004 leader_election.cc:341] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): Illegal state: cannot vote while shut down
I20260926 18:02:52.802187 11004 leader_election.cc:302] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da04736c877f463084a56916cfee8975; no voters: 0b81d04cd3474908a47f38ee64f48d1d, 7a1c9473d9234bca9c36f66cab316deb
I20260926 18:02:52.803560 11038 raft_consensus.cc:2454] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b81d04cd3474908a47f38ee64f48d1d in term 2.
I20260926 18:02:52.797750 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c49cb6f86f4bf791934acb0c8886c2" candidate_uuid: "0b81d04cd3474908a47f38ee64f48d1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a1c9473d9234bca9c36f66cab316deb"
I20260926 18:02:52.804065 11303 raft_consensus.cc:2726] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 18:02:52.804725 10927 leader_election.cc:302] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b81d04cd3474908a47f38ee64f48d1d, da04736c877f463084a56916cfee8975; no voters:
W20260926 18:02:52.805346 10930 leader_election.cc:341] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): Illegal state: cannot vote while shut down
I20260926 18:02:52.806058 11298 raft_consensus.cc:2781] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 FOLLOWER]: Leader election won for term 2
I20260926 18:02:52.806535 11298 raft_consensus.cc:686] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 LEADER]: Becoming Leader. State: Replica: 0b81d04cd3474908a47f38ee64f48d1d, State: Running, Role: LEADER
I20260926 18:02:52.807214 11298 consensus_queue.cc:237] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } }
I20260926 18:02:52.814867 10784 catalog_manager.cc:5495] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d reported cstate change: term changed from 1 to 2, leader changed from 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132) to 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130). New cstate: current_term: 2 leader_uuid: "0b81d04cd3474908a47f38ee64f48d1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a1c9473d9234bca9c36f66cab316deb" member_type: VOTER last_known_addr { host: "127.6.90.132" port: 44363 } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:52.898926 10954 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260926 18:02:52.909895 11102 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:52.913496 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:52.913867 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 1 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 0 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 57c49cb6f86f4bf791934acb0c8886c2 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
da04736c877f463084a56916cfee8975 (127.6.90.131:46253): RUNNING
0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107): RUNNING [LEADER]
7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): not running
State: FAILED
Data state: TABLET_DATA_READY
Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
A = da04736c877f463084a56916cfee8975
B = 0b81d04cd3474908a47f38ee64f48d1d
C = 7a1c9473d9234bca9c36f66cab316deb
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260926 18:02:53.136953 6506 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260926 18:02:53.317875 11038 raft_consensus.cc:1261] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Refusing update from remote peer 0b81d04cd3474908a47f38ee64f48d1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260926 18:02:53.319016 11333 consensus_queue.cc:1035] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260926 18:02:53.329567 10930 consensus_queue.cc:579] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: The tablet replica hosted on peer 7a1c9473d9234bca9c36f66cab316deb has failed
W20260926 18:02:53.330056 10930 consensus_peers.cc:487] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d -> Peer 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): Couldn't send request to peer 7a1c9473d9234bca9c36f66cab316deb. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:53.337131 10964 consensus_queue.cc:237] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } }
I20260926 18:02:53.340463 11038 raft_consensus.cc:1261] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Refusing update from remote peer 0b81d04cd3474908a47f38ee64f48d1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 18:02:53.341543 11336 consensus_queue.cc:1035] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 18:02:53.345721 11337 raft_consensus.cc:2932] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } }
I20260926 18:02:53.346830 11038 raft_consensus.cc:2932] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } }
I20260926 18:02:53.351783 10774 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 57c49cb6f86f4bf791934acb0c8886c2 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260926 18:02:53.354305 10784 catalog_manager.cc:5495] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d reported cstate change: config changed from index -1 to 3, VOTER 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132) evicted. New cstate: current_term: 2 leader_uuid: "0b81d04cd3474908a47f38ee64f48d1d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260926 18:02:53.364148 11102 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c49cb6f86f4bf791934acb0c8886c2 with delete_type TABLET_DATA_TOMBSTONED (TS 7a1c9473d9234bca9c36f66cab316deb not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:51132
I20260926 18:02:53.366153 11343 ts_tablet_manager.cc:1905] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260926 18:02:53.367259 10964 consensus_queue.cc:237] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } }
I20260926 18:02:53.372367 11038 raft_consensus.cc:1261] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Refusing update from remote peer 0b81d04cd3474908a47f38ee64f48d1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260926 18:02:53.374043 11333 consensus_queue.cc:1035] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260926 18:02:53.380321 11343 ts_tablet_manager.cc:1918] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260926 18:02:53.379776 11336 raft_consensus.cc:2932] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 07d8fe4d5993476fa862374ffa11b2b4 (127.6.90.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } } }
I20260926 18:02:53.380918 11343 log.cc:1196] T 57c49cb6f86f4bf791934acb0c8886c2 P 7a1c9473d9234bca9c36f66cab316deb: Deleting WAL directory at /tmp/dist-test-taskD6zog7/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1790445738975945-6506-0/minicluster-data/ts-3-root/wals/57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:53.381534 11038 raft_consensus.cc:2932] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, NON_VOTER 07d8fe4d5993476fa862374ffa11b2b4 (127.6.90.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } } }
I20260926 18:02:53.383220 10774 catalog_manager.cc:4841] TS 7a1c9473d9234bca9c36f66cab316deb (127.6.90.132:44363): tablet 57c49cb6f86f4bf791934acb0c8886c2 (table test-workload [id=4dacbc06f9924943bdb9a2f4a575783f]) successfully deleted
I20260926 18:02:53.388029 10774 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 57c49cb6f86f4bf791934acb0c8886c2 with cas_config_opid_index 3: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20260926 18:02:53.388597 10930 consensus_peers.cc:487] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d -> Peer 07d8fe4d5993476fa862374ffa11b2b4 (127.6.90.129:46411): Couldn't send request to peer 07d8fe4d5993476fa862374ffa11b2b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57c49cb6f86f4bf791934acb0c8886c2. This is attempt 1: this message will repeat every 5th retry.
I20260926 18:02:53.391610 10784 catalog_manager.cc:5495] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d reported cstate change: config changed from index 3 to 4, NON_VOTER 07d8fe4d5993476fa862374ffa11b2b4 (127.6.90.129) added. New cstate: current_term: 2 leader_uuid: "0b81d04cd3474908a47f38ee64f48d1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260926 18:02:53.685494 10879 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:53.707911 11102 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:53.709178 11028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260926 18:02:53.713709 10954 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
44636d5fbfa948e88f08cc2708b0d9f5 | 127.6.90.190:37829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
07d8fe4d5993476fa862374ffa11b2b4 | 127.6.90.129:46411 | HEALTHY | <none> | 0 | 0
0b81d04cd3474908a47f38ee64f48d1d | 127.6.90.130:33107 | HEALTHY | <none> | 1 | 0
7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | HEALTHY | <none> | 0 | 0
da04736c877f463084a56916cfee8975 | 127.6.90.131:46253 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Small | 7a1c9473d9234bca9c36f66cab316deb | 127.6.90.132:44363 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260926 18:02:53.913298 6506 log_verifier.cc:126] Checking tablet 57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:53.944896 11375 ts_tablet_manager.cc:927] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Initiating tablet copy from peer 0b81d04cd3474908a47f38ee64f48d1d (127.6.90.130:33107)
I20260926 18:02:53.946229 11375 tablet_copy_client.cc:316] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: tablet copy: Beginning tablet copy session from remote peer at address 127.6.90.130:33107
I20260926 18:02:53.948062 6506 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:53.953785 6506 tablet_server.cc:178] TabletServer@127.6.90.129:0 shutting down...
I20260926 18:02:53.961925 10974 tablet_copy_service.cc:140] P 0b81d04cd3474908a47f38ee64f48d1d: Received BeginTabletCopySession request for tablet 57c49cb6f86f4bf791934acb0c8886c2 from peer 07d8fe4d5993476fa862374ffa11b2b4 ({username='slave'} at 127.0.0.1:54624)
I20260926 18:02:53.963485 10974 tablet_copy_service.cc:161] P 0b81d04cd3474908a47f38ee64f48d1d: Beginning new tablet copy session on tablet 57c49cb6f86f4bf791934acb0c8886c2 from peer 07d8fe4d5993476fa862374ffa11b2b4 at {username='slave'} at 127.0.0.1:54624: session id = 07d8fe4d5993476fa862374ffa11b2b4-57c49cb6f86f4bf791934acb0c8886c2
I20260926 18:02:53.971098 10974 tablet_copy_source_session.cc:205] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 18:02:53.973865 11375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c49cb6f86f4bf791934acb0c8886c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 18:02:53.974368 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:53.983536 11375 tablet_copy_client.cc:791] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: tablet copy: Starting download of 0 data blocks...
I20260926 18:02:53.983939 11375 tablet_copy_client.cc:655] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: tablet copy: Starting download of 1 WAL segments...
I20260926 18:02:53.986835 11375 tablet_copy_client.cc:531] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 18:02:53.992268 11375 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Bootstrap starting.
I20260926 18:02:54.007742 11375 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 18:02:54.008378 11375 tablet_bootstrap.cc:492] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Bootstrap complete.
I20260926 18:02:54.008862 11375 ts_tablet_manager.cc:1397] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Time spent bootstrapping tablet: real 0.017s user 0.020s sys 0.000s
I20260926 18:02:54.010663 11375 raft_consensus.cc:348] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } }
I20260926 18:02:54.011224 11375 raft_consensus.cc:729] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 07d8fe4d5993476fa862374ffa11b2b4, State: Initialized, Role: LEARNER
I20260926 18:02:54.011656 11375 consensus_queue.cc:260] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da04736c877f463084a56916cfee8975" member_type: VOTER last_known_addr { host: "127.6.90.131" port: 46253 } } peers { permanent_uuid: "0b81d04cd3474908a47f38ee64f48d1d" member_type: VOTER last_known_addr { host: "127.6.90.130" port: 33107 } } peers { permanent_uuid: "07d8fe4d5993476fa862374ffa11b2b4" member_type: NON_VOTER last_known_addr { host: "127.6.90.129" port: 46411 } attrs { promote: true } }
I20260926 18:02:54.013477 11375 ts_tablet_manager.cc:1428] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 18:02:54.014830 10974 tablet_copy_service.cc:342] P 0b81d04cd3474908a47f38ee64f48d1d: Request end of tablet copy session 07d8fe4d5993476fa862374ffa11b2b4-57c49cb6f86f4bf791934acb0c8886c2 received from {username='slave'} at 127.0.0.1:54624
I20260926 18:02:54.015120 10974 tablet_copy_service.cc:433] P 0b81d04cd3474908a47f38ee64f48d1d: ending tablet copy session 07d8fe4d5993476fa862374ffa11b2b4-57c49cb6f86f4bf791934acb0c8886c2 on tablet 57c49cb6f86f4bf791934acb0c8886c2 with peer 07d8fe4d5993476fa862374ffa11b2b4
I20260926 18:02:54.017051 6506 tablet_replica.cc:330] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4: stopping tablet replica
I20260926 18:02:54.017521 6506 raft_consensus.cc:2229] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4 [term 2 LEARNER]: Raft consensus shutting down.
I20260926 18:02:54.017875 6506 raft_consensus.cc:2258] T 57c49cb6f86f4bf791934acb0c8886c2 P 07d8fe4d5993476fa862374ffa11b2b4 [term 2 LEARNER]: Raft consensus is shut down!
I20260926 18:02:54.035394 6506 tablet_server.cc:194] TabletServer@127.6.90.129:0 shutdown complete.
I20260926 18:02:54.043892 6506 tablet_server.cc:178] TabletServer@127.6.90.130:0 shutting down...
I20260926 18:02:54.058630 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:54.059147 6506 tablet_replica.cc:330] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d: stopping tablet replica
I20260926 18:02:54.059594 6506 raft_consensus.cc:2229] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 LEADER]: Raft consensus shutting down.
I20260926 18:02:54.060297 6506 raft_consensus.cc:2258] T 57c49cb6f86f4bf791934acb0c8886c2 P 0b81d04cd3474908a47f38ee64f48d1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:54.078768 6506 tablet_server.cc:194] TabletServer@127.6.90.130:0 shutdown complete.
I20260926 18:02:54.086498 6506 tablet_server.cc:178] TabletServer@127.6.90.131:0 shutting down...
I20260926 18:02:54.102654 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:54.103200 6506 tablet_replica.cc:330] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975: stopping tablet replica
I20260926 18:02:54.103653 6506 raft_consensus.cc:2229] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 18:02:54.104125 6506 raft_consensus.cc:2258] T 57c49cb6f86f4bf791934acb0c8886c2 P da04736c877f463084a56916cfee8975 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:54.123485 6506 tablet_server.cc:194] TabletServer@127.6.90.131:0 shutdown complete.
I20260926 18:02:54.131734 6506 tablet_server.cc:178] TabletServer@127.6.90.132:0 shutting down...
I20260926 18:02:54.147351 6506 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 18:02:54.163654 6506 tablet_server.cc:194] TabletServer@127.6.90.132:0 shutdown complete.
I20260926 18:02:54.171496 6506 master.cc:537] Master@127.6.90.190:37829 shutting down...
I20260926 18:02:54.184458 6506 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 18:02:54.184894 6506 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 18:02:54.185134 6506 tablet_replica.cc:330] T 00000000000000000000000000000000 P 44636d5fbfa948e88f08cc2708b0d9f5: stopping tablet replica
I20260926 18:02:54.202371 6506 master.cc:552] Master@127.6.90.190:37829 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (4843 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (9331 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (35097 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260926 18:02:54.232059 6506 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~10 seconds ago
I20260926 18:02:54.232282 6506 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4584: suppressed but not reported on 4 messages since previous log ~14 seconds ago
I20260926 18:02:54.232507 6506 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 23 messages since previous log ~15 seconds ago
I20260926 18:02:54.232649 6506 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 117 messages since previous log ~26 seconds ago
I20260926 18:02:54.232856 6506 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 28 messages since previous log ~32 seconds ago